-- Logs begin at Fri 2022-09-02 20:22:50 UTC, end at Fri 2022-09-02 21:47:14 UTC. -- Sep 02 21:01:00.186881 dv-cinder-852211-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 02 21:01:00.199522 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 2 21:01:00 2022] *** Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: nodename: dv-cinder-852211-9-iscsi Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: machine: x86_64 Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: clock source: unix Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: pcre jit disabled Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: detected number of CPU cores: 4 Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: current working directory: / Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: detected binary path: /usr/bin/uwsgi-core Sep 02 21:01:00.203666 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: your processes number limit is 47680 Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: your memory page size is 4096 bytes Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: detected max file descriptor number: 2048 Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: lock engine: pthread robust mutexes Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: thunder lock: enabled Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 02 21:01:00.204336 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 02 21:01:00.248784 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: Python main interpreter initialized at 0x5651444c09c0 Sep 02 21:01:00.248784 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: python threads support enabled Sep 02 21:01:00.248784 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: your server socket listen backlog is limited to 100 connections Sep 02 21:01:00.248784 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: your mercy for graceful operations on workers is 90 seconds Sep 02 21:01:00.249310 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: mapped 403077 bytes (393 KB) for 2 cores Sep 02 21:01:00.249310 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: *** Operational MODE: preforking *** Sep 02 21:01:00.249587 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: *** uWSGI is running in multiple interpreter mode *** Sep 02 21:01:00.249587 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: spawned uWSGI master process (pid: 81832) Sep 02 21:01:00.249914 dv-cinder-852211-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 02 21:01:00.250985 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: spawned uWSGI worker 1 (pid: 81842, cores: 1) Sep 02 21:01:00.250985 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: spawned uWSGI worker 2 (pid: 81843, cores: 1) Sep 02 21:01:00.250985 dv-cinder-852211-9-iscsi devstack@keystone.service[81832]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 02 21:01:05.056143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 21:01:05.056143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: from cryptography.utils import int_from_bytes Sep 02 21:01:05.057055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 21:01:05.057055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: from cryptography.utils import int_from_bytes Sep 02 21:01:05.062285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 21:01:05.062285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: from cryptography.utils import int_from_bytes Sep 02 21:01:05.063201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 21:01:05.063201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: from cryptography.utils import int_from_bytes Sep 02 21:01:05.117282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 21:01:05.117560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 21:01:05.117696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 21:01:05.117908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 21:01:05.118062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] command line args: [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 21:01:05.118197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] command line args: [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 21:01:05.118339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 21:01:05.118482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 21:01:05.118482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ================================================================================ {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 21:01:05.118730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ================================================================================ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 21:01:05.118903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] admin_token = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] admin_token = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] config_dir = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_dir = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] config_source = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.119588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] debug = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.120152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.120282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.126615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] config_source = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] debug = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.127664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_config_append = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_dir = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.128941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_file = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.129793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_config_append = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_dir = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.130965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.131830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.132995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.99/identity {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] publish_errors = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.133782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.134867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] standard_threads = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] use_journal = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.135898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] use_json = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] use_stderr = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] use_syslog = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.99/identity {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.136742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] publish_errors = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.136871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.136871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.137122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.137253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.137392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.137517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.137645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.137903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.137903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.138440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.138567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.138691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.138829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.138829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.139077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] standard_threads = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.139217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.139347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.139476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.139476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.139707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.139850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.139983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.140111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.140251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_journal = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.140391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.140528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_json = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.140668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.140668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.140918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_stderr = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.141048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.141176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] use_syslog = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.141306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.141454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 21:01:05.141587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.141587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.141876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.142773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.143981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.144268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.144410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.144410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.144730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.144862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.145926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.146897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.147801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.148937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.149242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.149411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.149629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.149858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.149858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.150851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.151947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.152829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.153547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.155251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.155444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.155444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.155779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.155977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.156774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.157836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.158905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.159982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.160750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.161932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.162924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.external = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.163951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.password = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] auth.token = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.164864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.165937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.166807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.167821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.168966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.169677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.170966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.171994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.172898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.173926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.174942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.175908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.176948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.177744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.178821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.179916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.180801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.external = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.181979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.182979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.password = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] auth.token = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.183884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.184982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.185944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.186978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.187875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.188987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.189913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.190914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.191938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.192965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.193908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] role.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] role.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.194939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.195937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.196975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.197832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.198893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.199958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.200198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.201516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.201672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.202366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.202366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.202620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.202736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.202736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.203965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.204911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.205971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.connection = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.206912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.207859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.208869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.209942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.210182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.210313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.210313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81842) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.212710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.214759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.215069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.215444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.215804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.217045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.217285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.217589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.217999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.218357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.218741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.219117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.219495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.219872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.220252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.220602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.220978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.221340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.221699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.222113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.222446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.222819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.223192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.223555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.223940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.224292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.224667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.225026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.225387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.225775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.226146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.226511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.226867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.227226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.227596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.227976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.228351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.228713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.229090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.229945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.229945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.230204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.230645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.231074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.231585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.232052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.232477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.232921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.233370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.233847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.234307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.234750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.235194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.235637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.236106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.236526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.237024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.237471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.237977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.238421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.239147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.239571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.242917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.244528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.245584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.245584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.245584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.245584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.245584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.246995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.247716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.248181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.249383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.249612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.249864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.249864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.250183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.250336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.250578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.250814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.251056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.251293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.251530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.251774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.252012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.252258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.252497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.252745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.252981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.253354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.253797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.254206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.254559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 21:01:05.254895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 21:01:05.260441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.282100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.282363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.282531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.291081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.291527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.291823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.310943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.311157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.351102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.357640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.406942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.407173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.409311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.409545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.566887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.567893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.568035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.568171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.568294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.568424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.598254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.598548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.598730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.598890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.599877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.600034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.600034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.600343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.600343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.621215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.621402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.642566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.642780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.642915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.643940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.644023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.644286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.644385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.644629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.644712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81842) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 21:01:05.673529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.673739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.673871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 21:01:05.673985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 21:01:05.674399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.674486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.674709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 21:01:05.674794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 21:01:05.675138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.675224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.675546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.675631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.675950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.676034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.676355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.676439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.694188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.694364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.694628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.694740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.695045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.695129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.695482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.695567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.708034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.708225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.713115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.713228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.713563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.713648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.714025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.714111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.714464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.714549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.714904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.715000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.715315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.715397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.715740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.715822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.716193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.716332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.716637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.716780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.717088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.717216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.717526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.717645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.759641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.759848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.762220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.762379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.762672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.762772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.771227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.771425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.771686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.771773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.772127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.772300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.783714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.783910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.784179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.784270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.794515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.796980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.797097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.797778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.798045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.798045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.798172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.798172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.798500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.826053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.826294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.826442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.826603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.833460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.833658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.839429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.839624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.839922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.840160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.848008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.848224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.853827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.854051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.860451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.860691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.860946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.861035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.861395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.861484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.861902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.861994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.862342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.862444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.862792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.862879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.893788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.894000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.894265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.894365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.894687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.894775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.895104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.895275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.907773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.907989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.910684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.910855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.913551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.913846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.914037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.914140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.919467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.919681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.919956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.920064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.920402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.920503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.920840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.920946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.921457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.921557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.943083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.943292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.943547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.946856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.975890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.976100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.981236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.981426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.986822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.987111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.989587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.989760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.992361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.992780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:05.992872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.992948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.993424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:05.993510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.005742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.005986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.008331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.008466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.014141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.014406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.014785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.014917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.015228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.015354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.015655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.015793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.029781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.029981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.030260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.030346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.031987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.032075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.042965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81842) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.043165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 21:01:06.043425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.043512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.043873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.044078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.044360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.044752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.044902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.045182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.045327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.045606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.045812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.046064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.046220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.046492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.046623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.046925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.047059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.047366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.047511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.047795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.047942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.048209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.048351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.048627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.048770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.049036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81842) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 21:01:06.110275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5651444c09c0 pid: 81842 (default app) Sep 02 21:01:06.110908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5651444c09c0 pid: 81843 (default app) Sep 02 21:01:06.114150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76867-ecaf-4e8e-957b-8b648a05a495 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:06.130846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76867-ecaf-4e8e-957b-8b648a05a495 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:06.130846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde76867-ecaf-4e8e-957b-8b648a05a495 None None] PATH_INFO: `/v3` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:06.130846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1/1] 10.222.0.99 () {50 vars in 791 bytes} [Fri Sep 2 21:01:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:14.063193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:14.063772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:14.063772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:14.067986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1/2] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:14.073383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:14.073878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:14.073979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:14.146237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: INFO dbcounter [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] Registered counter for database keystone Sep 02 21:01:14.167985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG oslo_db.sqlalchemy.engines [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81842) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 21:01:14.174746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writer thread running {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 21:01:14.358807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG passlib.handlers.bcrypt [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81842) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 21:01:14.359113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG passlib.handlers.bcrypt [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81842) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 21:01:14.418668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:14.420412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:14.561707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 2/3] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 02 21:01:15.053603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:15.125060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: INFO dbcounter [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] Registered counter for database keystone Sep 02 21:01:15.144723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG oslo_db.sqlalchemy.engines [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81843) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 21:01:15.151110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writer thread running {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 21:01:15.272958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154d90e-7c47-4423-81e9-6b40ab2bb11b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:15.277450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:15.278230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:15.278594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:15.278960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:15.287475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:15.340735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 21:01:15.341003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ecc3c-d28d-4bda-90f7-f73cb83af801 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:15.351212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 2/4] 10.222.0.99 () {58 vars in 1324 bytes} [Fri Sep 2 21:01:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:01:15.357743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:15.372668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e79b3-0bf8-4784-8ab8-767dbd51237b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:15.377425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:15.378603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:15.378880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:15.379145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] PATH_INFO: `/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:15.380741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] RBAC: Authorizing `identity:get_project(project_id=8077ff7239e7425ab3b0ae8d4fecdd2f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:15.435725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING keystone.common.rbac_enforcer.enforcer [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 21:01:15.436062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c86f10-778d-4051-be0b-3bf9053555e2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:15.440792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 3/5] 10.222.0.99 () {56 vars in 1207 bytes} [Fri Sep 2 21:01:15 2022] GET /identity/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:17.365093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b6c0f-23d3-4c17-b601-f21781c2df0c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:17.365664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b6c0f-23d3-4c17-b601-f21781c2df0c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:17.365664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b6c0f-23d3-4c17-b601-f21781c2df0c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:17.370013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 3/6] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:17.378145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:17.378711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:17.378711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:17.401967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:17.404420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:17.413081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 4/7] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 02 21:01:17.687114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b6c0f-23d3-4c17-b601-f21781c2df0c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:17.699615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b6c0f-23d3-4c17-b601-f21781c2df0c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:17.703996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:17.704653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:17.704962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:17.705261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:17.713655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:17.715519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12a6e5-a094-4bb6-888b-42f3f529ecad admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:17.724803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 4/8] 10.222.0.99 () {58 vars in 1324 bytes} [Fri Sep 2 21:01:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:01:17.730634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:17.740714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225da8e-c0a8-4bb8-aaa7-188b1bd054c4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:17.745245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:17.745915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:17.746192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:17.746457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:17.748911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:17.750806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81aeb10-cfe3-4222-9865-f06760e94413 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:17.752358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 5/9] 10.222.0.99 () {56 vars in 1198 bytes} [Fri Sep 2 21:01:17 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:19.895507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75070-81f3-4c11-b90f-de788c04ed55 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:19.896022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75070-81f3-4c11-b90f-de788c04ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:19.896022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75070-81f3-4c11-b90f-de788c04ed55 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:19.898204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 5/10] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:19.899579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c90303-c014-4eb1-9d5f-8ce0627cf86d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:19.900171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c90303-c014-4eb1-9d5f-8ce0627cf86d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:19.900171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c90303-c014-4eb1-9d5f-8ce0627cf86d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:19.901968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 6/11] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:19.904430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:19.904842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:19.905230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:19.909965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:19.910353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:19.910627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:19.931091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:19.932316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:19.961992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG passlib.handlers.bcrypt [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81843) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 21:01:19.962448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG passlib.handlers.bcrypt [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81843) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 21:01:20.025378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:20.027406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:20.036221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 7/12] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:20.176442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 6/13] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:20.320544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.348337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:20.436893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b427c64-6bdf-4d19-99d9-4f9955dc55dc None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.471478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.476127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:20.476791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:20.477066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:20.477335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:20.491515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:20.501179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:20.503000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:20.504953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 8/14] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:20.509032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.598970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8e0e-ad95-42e4-aab8-77aca2c683c1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.603496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:20.612002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:20.634979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3728b-fabf-40ef-ab17-2af9fe0e1535 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.647771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:20.649347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:20.649899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:20.650396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:20.675881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:20.676530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:20.676807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:20.677070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:20.677758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:20.678059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:20.679459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:20.693601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 9/15] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:20.699217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.738650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:20.740343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:20.824389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4568e49-3d46-4b81-96e2-b3e8689bd7cd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.848898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 7/16] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:01:20.854614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.857983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:20.861793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:20.862434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:20.862735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:20.863018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:20.864283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:20.865621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:20.867457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 10/17] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:20.873391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:20.890471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:20.962228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727be4b-c117-4f45-99ff-700f3d53dc18 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.979289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d8499-7948-41d2-9a3c-629ef10b55f8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:20.998250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:21.011842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:21.012275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:21.013033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:21.015119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:21.015774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:21.016051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:21.016318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:21.017452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:21.018965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:21.020520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 11/18] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:21.033906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:21.046700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:21.048197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:01:21.069349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:21.071072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:01:21.072440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:01:21.072440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:01:21.072440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:21.072440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 8/19] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:01:21.082170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:21.138934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:21.148637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:21.159062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d23c20-eec0-49d9-8ce8-e85fceb3939a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:21.195939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b0a01-2a81-4e24-b565-15a017e7088e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:21.196952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:21.197570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:21.197878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:21.198153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:21.198938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.domain.id=default, user.id=65f8e87a284d46c3becdb8085954efaf) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:21.199241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:21.202005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806ce91-c8e7-4bb3-8915-b495ba3e9d0a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:21.209169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 12/20] 10.222.0.99 () {56 vars in 1322 bytes} [Fri Sep 2 21:01:21 2022] GET /identity/v3/role_assignments?user.id=65f8e87a284d46c3becdb8085954efaf&scope.domain.id=default&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 233 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:21.235128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:21.235768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:21.236045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:21.236316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:21.236985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:21.237288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:21.239116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1948-14ec-49c4-b9e6-71cd2b860c6f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:21.261042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 9/21] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:23.867750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c644-9685-44df-9696-00530f73adf9 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:23.868679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c644-9685-44df-9696-00530f73adf9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:23.869170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c644-9685-44df-9696-00530f73adf9 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:23.870796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 13/22] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:23.877083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c11ed-6fba-4999-9def-602f8f6c5e31 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:23.877572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c11ed-6fba-4999-9def-602f8f6c5e31 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:23.877665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c11ed-6fba-4999-9def-602f8f6c5e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:23.900164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-658c11ed-6fba-4999-9def-602f8f6c5e31 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:23.901398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-658c11ed-6fba-4999-9def-602f8f6c5e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:24.010612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e43ca-85e1-4e87-b98b-39e9c5711ff6 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.011245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e43ca-85e1-4e87-b98b-39e9c5711ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.011711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e43ca-85e1-4e87-b98b-39e9c5711ff6 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.014037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 14/23] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:24.020580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b38605-257d-46c2-a0dd-7770e5ebe462 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.021309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b38605-257d-46c2-a0dd-7770e5ebe462 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.021910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b38605-257d-46c2-a0dd-7770e5ebe462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.025304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 10/24] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:24.050756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e4b38605-257d-46c2-a0dd-7770e5ebe462 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:24.052318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e4b38605-257d-46c2-a0dd-7770e5ebe462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:24.052822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdeef-deff-4a0d-b313-b02b710db16d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.053108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdeef-deff-4a0d-b313-b02b710db16d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.053383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdeef-deff-4a0d-b313-b02b710db16d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.055054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 11/25] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:24.060813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951240c3-871e-49d5-b395-672b2a021655 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.061089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951240c3-871e-49d5-b395-672b2a021655 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.061540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951240c3-871e-49d5-b395-672b2a021655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.097649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-951240c3-871e-49d5-b395-672b2a021655 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:24.099112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-951240c3-871e-49d5-b395-672b2a021655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:24.156545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 15/26] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:24.158379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472026-95ca-4f2f-a2c4-59cc1da661e9 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.158836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472026-95ca-4f2f-a2c4-59cc1da661e9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.159239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe472026-95ca-4f2f-a2c4-59cc1da661e9 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.167576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 16/27] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:24.179237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.179976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.180531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.185517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:24.218428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 12/28] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:24.221800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:24.223402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:24.336366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951240c3-871e-49d5-b395-672b2a021655 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:24.343611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 17/29] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:24.477302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951240c3-871e-49d5-b395-672b2a021655 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:24.514871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:24.515500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.515775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.516045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.516790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:24.518269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:24.530782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:24.540585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:24.762449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dba34d-047d-430f-ad23-31e45efc2feb None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:24.767819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 13/30] 10.222.0.99 () {60 vars in 1131 bytes} [Fri Sep 2 21:01:24 2022] POST /identity/v3/roles => generated 225 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:24.770374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:24.827996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:24.890540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:24.891297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.891709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.891995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.917555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7cbe7-1296-4b64-b3da-92c15db12d41 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:24.920467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:24.930217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:01:24.959426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:24.960311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:01:24.961091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:01:24.961091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:01:24.961091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:24.961091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 18/31] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:01:24.961696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:24.964513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:24.965983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:24.967438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:24.967916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:24.968876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:24.970796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.040799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:25.058975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 14/32] 10.222.0.99 () {60 vars in 1131 bytes} [Fri Sep 2 21:01:24 2022] POST /identity/v3/roles => generated 219 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:25.060861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.099200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39539400-b81b-4f05-9c9c-12b9f0f4b212 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.143131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.143785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.144255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.144543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.162583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:25.172311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:25.173224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:25.173997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:25.175450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 19/33] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:25.189959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.299574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:25.306384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7af8f-cef9-4e4f-b366-e07ff3953a30 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.321994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eaa7e-218c-4b67-a39d-eac40b0e028c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.346101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:25.366351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.367477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.367985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.368457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.369682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:25.370188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:25.372005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.373406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.374657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.375245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.375997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.377090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:25.377920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:25.382574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.388310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 15/34] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:25.393447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.393812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 20/35] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:25.400564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.557508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5867-3642-4981-8624-bead4ce0979d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.564530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b797-689f-41de-87ec-328da3c0befe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.598098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.598745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.599021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.599295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.600453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:25.601944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bac5d5-c57c-4b0c-b0b2-4856bc9f501a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.603716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 21/36] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:25 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:25.607699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aff86-d2d2-40c2-883c-98a99907e47a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.607699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aff86-d2d2-40c2-883c-98a99907e47a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.607699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aff86-d2d2-40c2-883c-98a99907e47a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.607699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 22/37] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:25.612571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aff86-d2d2-40c2-883c-98a99907e47a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.617614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.618533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.619065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.619503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.620589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:25.622906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8336b-8aea-44f3-8db7-7dca5e65e2e3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.665611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:25.729455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aff86-d2d2-40c2-883c-98a99907e47a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.766283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:25.766920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.767176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.767441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.768558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:25.770078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fac00-426e-4455-bbf3-2f1db0f80e45 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:25.771765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 23/38] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:25.773004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ba03-1ebf-426e-9fdf-af9e3ba4304f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.773288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ba03-1ebf-426e-9fdf-af9e3ba4304f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.774681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ba03-1ebf-426e-9fdf-af9e3ba4304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.777613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 16/39] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:01:25 2022] POST /identity/v3/projects => generated 301 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:25.779168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842517-410e-4bb1-a8e2-2b074486c66d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.779457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842517-410e-4bb1-a8e2-2b074486c66d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.779730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8842517-410e-4bb1-a8e2-2b074486c66d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.781566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 17/40] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:25.785769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8842517-410e-4bb1-a8e2-2b074486c66d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:25.796899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d450ba03-1ebf-426e-9fdf-af9e3ba4304f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:25.798193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:25.798333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d450ba03-1ebf-426e-9fdf-af9e3ba4304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:25.909767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 24/41] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:25.909767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:25.909767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:25.909767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:25.932413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:25.933632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:25.954326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8842517-410e-4bb1-a8e2-2b074486c66d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:25.993359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:26.004969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:26.004969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:26.004969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:26.004969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] PATH_INFO: `/v3/domains/default/users/65f8e87a284d46c3becdb8085954efaf/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:26.013218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=65f8e87a284d46c3becdb8085954efaf, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:26.013218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:26.034690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 25/42] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:26.125777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 18/43] 10.222.0.99 () {58 vars in 1361 bytes} [Fri Sep 2 21:01:25 2022] PUT /identity/v3/domains/default/users/65f8e87a284d46c3becdb8085954efaf/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:26.186196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:26.314542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:26.316296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e283f0-2249-4745-960c-e625c2e536bd None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:26.366211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:26.366853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:26.367128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:26.367482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:26.368733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:26.370727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6aa0a-5703-4052-a47d-194559e58843 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:26.403398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:26.440918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090ee4f-4018-4b94-97bd-682ac6bc9536 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:26.479185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:26.479831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:26.480109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:26.480377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:26.481581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:26.483100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:26.485052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 19/44] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:26.490541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:26.638969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e435e36-8f24-44ea-b2b5-0f48ecd1af99 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:26.696288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:26.696883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:26.697166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:26.697443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:26.698184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:26.700265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01001acf-3355-4fe0-b6a1-acdc94ca1eba None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:26.707666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:26.736780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 26/45] 10.222.0.99 () {60 vars in 1131 bytes} [Fri Sep 2 21:01:26 2022] POST /identity/v3/roles => generated 223 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:26.869175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 20/46] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:01:26 2022] POST /identity/v3/projects => generated 312 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:28.138806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6266d8-cd9b-4d70-a732-1e08fc6d48db None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.139636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6266d8-cd9b-4d70-a732-1e08fc6d48db None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.139784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6266d8-cd9b-4d70-a732-1e08fc6d48db None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.141177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 27/47] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:28.147201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.147850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.147850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.172138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:28.173675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:28.248181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 21/48] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:28.526809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6266d8-cd9b-4d70-a732-1e08fc6d48db None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:28.568367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:28.651990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6266d8-cd9b-4d70-a732-1e08fc6d48db None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:28.705744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:28.706533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.706861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.707177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.721031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:28.726363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:28.727687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:28.728947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 28/49] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:28.734853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:28.845011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9a4c9-7b67-4f2e-97c8-4f65abb31369 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:28.862698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:28.879517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:28.880131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.881851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.881851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.881851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:28.881851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:28.883031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d13d-a0a1-4088-babd-69b9eeb9d254 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:28.895671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 22/50] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:28.902687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:28.906876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd58e1b-1dcc-4d65-9767-0376b58e32be None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.907164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd58e1b-1dcc-4d65-9767-0376b58e32be None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.907436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd58e1b-1dcc-4d65-9767-0376b58e32be None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.909126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 23/51] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:28.914035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:28.914319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:28.914589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:28.933088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:28.935630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:29.000205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:29.005024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 24/52] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:29.007301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8d5e2-26b7-4ce5-9620-fb16b3b8d74f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:29.044057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:29.044745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:29.045024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:29.045298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:29.046573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:29.048135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:29.049729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 29/53] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:28 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:29.055370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:29.156865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4c187-14c2-4642-82f3-73ca5ec942a3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:29.275076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:29.282818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:29.301337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:29.302019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:29.302295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:29.302790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:29.303691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:29.305164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:29.312319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 25/54] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:29.313894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:29.394122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667be47-92e4-42e0-b001-56e664ea3ef7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:29.424058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991178e7-9ddd-4c4b-858c-aacadf867add None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:29.430360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:29.431017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:29.431303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:29.432106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:29.433378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:29.437416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:29.439392 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 30/55] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:29.444666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:29.462207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:29.462816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:29.463093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:29.463366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:29.464191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.domain.id=default, user.id=65f8e87a284d46c3becdb8085954efaf) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:29.464400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:29.466598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084415ff-e3dc-46bf-aed0-ede54d971c92 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:29.479132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 26/56] 10.222.0.99 () {56 vars in 1322 bytes} [Fri Sep 2 21:01:29 2022] GET /identity/v3/role_assignments?user.id=65f8e87a284d46c3becdb8085954efaf&scope.domain.id=default&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:29.479362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:29.562981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519844ed-ad2f-47e8-bf34-c1e030863f47 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:29.601791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:29.602426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:29.602714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:29.602994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:29.603666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:29.608138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfadc75-ca3b-4516-af2a-a8e77490a995 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:29.756939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 31/57] 10.222.0.99 () {60 vars in 1140 bytes} [Fri Sep 2 21:01:29 2022] POST /identity/v3/projects => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:31.718127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cb49-167b-454c-ba6a-19454861c091 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:31.718636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cb49-167b-454c-ba6a-19454861c091 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:31.718723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9cb49-167b-454c-ba6a-19454861c091 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:31.720029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 27/58] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:31.725272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:31.725683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:31.725882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:31.745073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:31.746838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:31.814124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 32/59] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:32.084096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9cb49-167b-454c-ba6a-19454861c091 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:32.129956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:32.187278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9cb49-167b-454c-ba6a-19454861c091 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:32.221700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:32.222356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:32.222661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:32.222935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:32.224058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:32.225515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7142ee-31fe-4b9a-a39c-9ac10fea8d9f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:32.227248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 28/60] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:32.233121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:32.248424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:32.336777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581868b-d019-4661-968d-2de2c1ec5b72 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:32.374110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:32.374728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:32.375005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:32.375275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:32.375932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:32.377394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166ba26-002f-4ab1-9f00-4f5f19979706 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:32.547604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 33/61] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:01:32 2022] POST /identity/v3/users => generated 275 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:35.650475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c3b-7580-4bf4-b0ac-d3b37dc8017d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:35.650980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c3b-7580-4bf4-b0ac-d3b37dc8017d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:35.651088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59c3b-7580-4bf4-b0ac-d3b37dc8017d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:35.653234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 29/62] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:35.666969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3322-d1fe-429d-bed0-f09d66a2062c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:35.667453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3322-d1fe-429d-bed0-f09d66a2062c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:35.667524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3322-d1fe-429d-bed0-f09d66a2062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:35.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b9bf3322-d1fe-429d-bed0-f09d66a2062c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:35.693537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b9bf3322-d1fe-429d-bed0-f09d66a2062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:35.773893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:35.783645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 34/63] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:35.841375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679da24-a580-4a61-9380-15400d9e32fb None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:35.842004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679da24-a580-4a61-9380-15400d9e32fb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:35.842452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679da24-a580-4a61-9380-15400d9e32fb None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:35.843970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 30/64] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:35.849473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98553-0d68-40db-a05e-88898b6e75e0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:35.850019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98553-0d68-40db-a05e-88898b6e75e0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:35.850461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c98553-0d68-40db-a05e-88898b6e75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:35.881235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f0c98553-0d68-40db-a05e-88898b6e75e0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:35.882751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f0c98553-0d68-40db-a05e-88898b6e75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:35.972070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 35/65] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:36.080441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa6be-ad14-4cc6-944c-19cc8ad0ded7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.081067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa6be-ad14-4cc6-944c-19cc8ad0ded7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.081562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468fa6be-ad14-4cc6-944c-19cc8ad0ded7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.083143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 31/66] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:36.088732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc10f-7db2-4eee-bc23-8681d844da30 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.089287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc10f-7db2-4eee-bc23-8681d844da30 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.090351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cc10f-7db2-4eee-bc23-8681d844da30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.109966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a33cc10f-7db2-4eee-bc23-8681d844da30 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:36.111991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a33cc10f-7db2-4eee-bc23-8681d844da30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:36.113243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa6be-ad14-4cc6-944c-19cc8ad0ded7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.218773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 36/67] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:36.224577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14532-6002-48d0-a2f0-7425f777cb63 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.224577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14532-6002-48d0-a2f0-7425f777cb63 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.224577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14532-6002-48d0-a2f0-7425f777cb63 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.236928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 37/68] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:36 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:36.238867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.239166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.239443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.283645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:36.288302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:36.311099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468fa6be-ad14-4cc6-944c-19cc8ad0ded7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.333143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:36.349558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:36.364341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 38/69] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:36.365442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.397441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.398990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.399666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.400383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.435346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:36.451609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:36.453246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:36.454735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 32/70] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:36.463018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.518859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056d216-0fc0-4e7d-9d9a-7449817ce414 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.568564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.569206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.569483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.569773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.581788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:36.586912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:36.587771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 39/71] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:36.594003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.634224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a531eb-fec6-40af-88d7-10dca46d5876 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.646153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:36.685549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.686902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.687397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.687734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.688643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:36.689010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:36.690658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca4268-59ba-47f0-a2c2-e83fbb6867d4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:36.700323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 33/72] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:36.702580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f54e-144b-4a17-a6c8-f5b49b42c2ba None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.703342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f54e-144b-4a17-a6c8-f5b49b42c2ba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.703987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4f54e-144b-4a17-a6c8-f5b49b42c2ba None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.705979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 34/73] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:36.706804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f54e-144b-4a17-a6c8-f5b49b42c2ba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.714569 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d5478-f4f1-4877-af0b-8b2086ffdb43 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.739959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:36.759702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.760342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.760612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.761613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.775571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:36.781500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:36.782763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:36.784111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 40/74] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:36.788135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.819416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4f54e-144b-4a17-a6c8-f5b49b42c2ba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.855137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.855792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.856060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.856328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.857001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:36.857300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:36.858734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade847-f4d6-4ff1-bd77-8b3349daba10 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:36.867312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 35/75] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:36.868629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.868918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.869194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.889657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:36.890930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:36.903106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa6e47-14ad-4e5e-9075-e1f7c4fea9ad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:36.915950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:36.949321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:36.951064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:36.951064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:36.951064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:36.967175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:36.968786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:36.973964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 41/76] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:36.974863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:36.975472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 36/77] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:36.976590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.009002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.087960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b3e59-e602-4d15-9a47-bb93a1c9242c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.096903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beac24-6ffe-48ac-b151-29d6b1ab3e74 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.129499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.130193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.130475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.130749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.134588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.135209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.135513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.135790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.136467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:37.137442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.139042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.144392 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:37.153064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 37/78] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:37.153064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:37.153370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:37.154183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 42/79] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:37.154946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.186009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.252304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.271692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a15e87-75d3-44e7-a79a-339a32221f29 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.275620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661899a-f343-44ac-aca8-218aca21f3bd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.311692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.312336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.312611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.312887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.313983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.314115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.314593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.314872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.315140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.315568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.317160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 38/80] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:37.317954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.391817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.393277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.420458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.425653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfea69-6397-41d7-90e9-b288c10368ab None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.461741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.462400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.462681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.462951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.464108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.466467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.468078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 39/81] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:37.468847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.495212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.498420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 43/82] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:37.499288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.576119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25ccce-e28b-441e-b0b0-441bb94bd7da None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.601792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48767-1393-4dad-9439-a134db1abaf7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.612594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.613214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.613567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.613822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.614495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:37.614797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.616186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.625967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 40/83] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:37.626829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.635083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.650829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.650829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.650829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.650829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.651258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.651258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.651258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 44/84] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:37.651463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.717872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.734365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b54875-91e7-47af-87bc-eb2ce3dd8afd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.755899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426944c-9eee-47f6-97be-464fe8e9892f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:37.771831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.772455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.772735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.773012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.791564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:37.792127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:37.792406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:37.792678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:37.848990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.850682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.856728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.864298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:37.866006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:37.943584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:37.960598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 41/85] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:37.961376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:37.973251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 45/86] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:37.974147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.067619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bd733-f7ba-4bd6-af33-580da02f945c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.073958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.082158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b981dc3-ede6-4f0f-a14d-003f2055cc6a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.109902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.110570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.110847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.111121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.112443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.114099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.115811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 42/87] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.116567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.126144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.126790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.127067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.127344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.128504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.130604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.132191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 46/88] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:37 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.132784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.177445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.225970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60142de1-b262-462b-b197-9c73b1f3ccb3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.242676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993caa-d38d-4f7c-9856-113541d2abde None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.263291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.263916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.264200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.264475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.265128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.266647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.279169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.279806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.280085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.280357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.281492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.283370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.285012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 47/89] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.285806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.317265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.387839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c646cd2-a16e-4208-9217-6dda18655e4b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.422773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.423452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.423729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.423999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.425097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.426582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.428230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 48/90] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.428995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.456619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 43/91] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:01:38 2022] POST /identity/v3/users => generated 289 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:38.457485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.569237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.573399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6225236-e48a-4ed2-94cf-ba90f5889089 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.615183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f85553-5adc-45bf-8195-6b25c4c309b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.632339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.634112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.634721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.635008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.635333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.636006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:38.636320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.637950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.649942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 49/92] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.651185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.666525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.667406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.668372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.668779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.775188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.778442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.850568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9345c-05a2-46f5-a062-323a1f83015b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:38.899292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:38.900302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:38.900589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:38.900864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:38.901604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=65f8e87a284d46c3becdb8085954efaf) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:38.902809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:38.904508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:38.911533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 50/93] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/role_assignments?user.id=65f8e87a284d46c3becdb8085954efaf&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:38.912408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.945217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 44/94] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:38.948341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:38.954932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:38.968532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:39.094236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4c8b-8c47-474a-8ebe-b0dedd9a3dae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:39.138777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:39.140040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:39.140555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:39.141010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] PATH_INFO: `/v3/projects/52bc0b446e374669ba4bc06ffe71e784` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:39.143237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=52bc0b446e374669ba4bc06ffe71e784)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:39.144998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:39.147622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 45/95] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/projects/52bc0b446e374669ba4bc06ffe71e784 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:39.155185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:39.167158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc9e2-81ff-4c6f-9499-674d47bdbb41 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:39.253160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:39.266507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:39.267140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:39.267418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:39.267694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:39.268443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:39.268751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:39.270265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f230f97-fe72-4e8c-ab8c-889dc0b4c922 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:39.286360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2e761-3b9e-4a60-9002-f0b1075e7bd4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:39.287678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 51/96] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:38 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 259 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:39.335539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:39.336520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:39.336980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:39.337430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:39.338394 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=52bc0b446e374669ba4bc06ffe71e784, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:39.338911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:39.340625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4e949-a38d-446f-806e-bd56addc6780 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:39.347846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 46/97] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:39 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=52bc0b446e374669ba4bc06ffe71e784&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:41.298713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae060-ab04-4dec-a101-197a42419412 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.299293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae060-ab04-4dec-a101-197a42419412 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.299293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ae060-ab04-4dec-a101-197a42419412 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.300780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 52/98] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:41.306280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece9bf-db80-4104-a50c-3b7770d4e01e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.307265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece9bf-db80-4104-a50c-3b7770d4e01e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.307769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfece9bf-db80-4104-a50c-3b7770d4e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.331531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dfece9bf-db80-4104-a50c-3b7770d4e01e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:41.333127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dfece9bf-db80-4104-a50c-3b7770d4e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:41.342264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee76d-ec64-47e5-909c-6819bc540c04 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.342890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee76d-ec64-47e5-909c-6819bc540c04 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.343384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ee76d-ec64-47e5-909c-6819bc540c04 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.345196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 53/99] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:41.350278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531eb8-1d07-41de-bda8-c1d2c6a3eac4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.350934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531eb8-1d07-41de-bda8-c1d2c6a3eac4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.351414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18531eb8-1d07-41de-bda8-c1d2c6a3eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.379697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-18531eb8-1d07-41de-bda8-c1d2c6a3eac4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:41.382094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-18531eb8-1d07-41de-bda8-c1d2c6a3eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:41.509681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 47/100] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:41.540014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 54/101] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:41.675843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248a64-0c2f-4b98-9989-1a6fe90a598f None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.676513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248a64-0c2f-4b98-9989-1a6fe90a598f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.677002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9248a64-0c2f-4b98-9989-1a6fe90a598f None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.678956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 48/102] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:41.685241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:41.685543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:41.685849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:41.714310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:41.716208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:41.738838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:41.816003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 55/103] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:42.003298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9248a64-0c2f-4b98-9989-1a6fe90a598f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.052990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.125179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:42.188753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9248a64-0c2f-4b98-9989-1a6fe90a598f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:42.268855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:42.270021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.270735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.271231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.272902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:42.274647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:42.276532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 49/104] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:42.283254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.337793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad67b-3850-4379-bb0f-3f5ef7e0f3c2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:42.412549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:42.413626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.414158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.414654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.439228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:42.455566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:42.457137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:42.458671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:42.459694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96986c09-3da2-4b31-9903-122451d2130c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:42.463244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 56/105] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:01:42.465111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.490419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:42.514628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:42.515296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.515578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.516013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.516968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:42.518753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03974403-64a1-471f-aed4-c3eb74e94442 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:42.531863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:42.623165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7da0e-6fb7-497b-9fd9-173e88304bad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:42.655146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:42.655890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.656207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.656412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.668499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-52658e0e-2376-4755-aef0-5e5ff0042b79 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:42.673612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:42.674637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:42.675391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 57/106] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:42.676547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ed-bdbf-48b4-b675-eb15e2983bde None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.676813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ed-bdbf-48b4-b675-eb15e2983bde None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.677284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b79ed-bdbf-48b4-b675-eb15e2983bde None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.679353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 58/107] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:42.679896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b79ed-bdbf-48b4-b675-eb15e2983bde None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.767123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:42.816417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b79ed-bdbf-48b4-b675-eb15e2983bde None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 50/108] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:01:42 2022] POST /identity/v3/projects => generated 302 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e6ae4-7b0c-4816-b6e2-38fc2a86da9a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e6ae4-7b0c-4816-b6e2-38fc2a86da9a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e6ae4-7b0c-4816-b6e2-38fc2a86da9a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 51/109] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:42.849812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e6ae4-7b0c-4816-b6e2-38fc2a86da9a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:42.884210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:42.884983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.885256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.885589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.886461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:42.886880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:42.888496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1b494-c285-4aa4-8e67-5b5ed7263fb8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:42.903976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 59/110] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:42.905384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cc85-3f12-4b4c-ac43-a35ddb467dd2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:42.905675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cc85-3f12-4b4c-ac43-a35ddb467dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:42.905978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cc85-3f12-4b4c-ac43-a35ddb467dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:42.929178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9898cc85-3f12-4b4c-ac43-a35ddb467dd2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:42.930773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9898cc85-3f12-4b4c-ac43-a35ddb467dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:43.017148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 60/111] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:43.018872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.019153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.019421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.025988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e6ae4-7b0c-4816-b6e2-38fc2a86da9a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.042170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:43.044185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:43.067655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.068201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.068484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.068762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.069461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:43.069813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:43.071236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95401a-0eb5-46e4-9b9e-08251b1b6fc5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:43.090083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 52/112] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:43.092184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eafb5-ff9e-4229-9271-b519f19712d4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.092918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eafb5-ff9e-4229-9271-b519f19712d4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.093395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3eafb5-ff9e-4229-9271-b519f19712d4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.095077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 53/113] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:43.096112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3eafb5-ff9e-4229-9271-b519f19712d4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.142804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.155312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 61/114] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:43.158906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.203945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.298223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3eafb5-ff9e-4229-9271-b519f19712d4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.350602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.351309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.351599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.351882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.353362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:43.355780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0934016-283f-4e2a-9cf5-581071f54f88 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:43.358227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 54/115] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:43.361839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.362309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.362587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.391377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc0be6-1411-41aa-a374-bc90fa54fb8e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.393460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:43.395779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:43.433082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.433883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.434093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.434409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] PATH_INFO: `/v3/users/3943caa0d3d747869805423a9e86e709` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.450902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] RBAC: Authorizing `identity:get_user(user_id=3943caa0d3d747869805423a9e86e709)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:43.451904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:43.453631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 62/116] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/users/3943caa0d3d747869805423a9e86e709 => generated 289 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:43.459142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.499856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 55/117] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:43.500753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.549379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.560968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.609819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c63d32-a18b-4888-91b9-edfed6d981a1 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.640481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ab578-5efd-4a39-b297-88a62bea93ce None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.665295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.666254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.666254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.666522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.687767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:43.697937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.698989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.699964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.700430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.701928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:43.703243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:43.704299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:43.705196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:43.705795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 63/118] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:43.707280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:43.710545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 56/119] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:43.711793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.712315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.843822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c4d7d-da9d-4022-81f0-53ceaef7a061 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.853597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1dfcb-5afe-4fee-9d81-91865397aef2 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:43.865523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.869956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:43.885287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.885972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.886251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.886721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.897873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:43.898094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:43.898506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:43.898788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:43.899090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:43.903037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:43.904149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:43.904912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 57/120] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:43.906524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:43.984028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:43.985928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.040651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a7ea-b798-4547-860b-477874086410 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.084132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.084754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.085034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.085309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.086122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:44.086320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.087869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.099689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 58/121] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.100396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.138897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 64/122] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:44.138897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.153476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.190399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.292945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e89df-70be-4cc4-b27f-bf63149ca4fe None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.302970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471574-a379-4bc5-856e-4a2abb67d05c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.342575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.343649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.343649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.343872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.347048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.348092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.348377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.348655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/25a810e91f074496a4fe262fab43ea53/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.351213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, user_id=25a810e91f074496a4fe262fab43ea53, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.353107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.356635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:44.362586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:44.364006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:44.364784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 65/123] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:44.370001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.443726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.472435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 59/124] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:01:44 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/25a810e91f074496a4fe262fab43ea53/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:44.473496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.485856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3512-f381-4936-9da2-aadf54fe65fd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.534696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.535894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.536368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.536831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.537979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:44.538656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.540704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.553261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 66/125] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.554826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.624166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.630643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c019cc-8c3f-48bb-87e1-22f01ee9969c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.678042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.679081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.679361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.679632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.680822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.682936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.685415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 60/126] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.691459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.702823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48443b-022a-4e54-84f2-c6b78aa92869 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.737990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.750727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.751378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.751663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.751945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.754677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.756162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.761756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 67/127] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.761756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.850511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afa85-eba7-4f34-8442-13f1f8e9c06c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.909583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.910633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.911118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.911653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.912532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:44.913195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.915037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.915785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b74bc5-f936-49f8-96d2-f220535f863c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:44.925360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 61/128] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.926598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:44.965350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:44.967180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:44.967801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:44.968280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:44.969992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:44.971868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:44.972956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:44.974221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 68/129] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:44.975577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.061148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322e020-4326-4c96-9d2a-907504de5701 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.076286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:45.100977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.101626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.101939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.102223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] PATH_INFO: `/v3/projects/52bc0b446e374669ba4bc06ffe71e784` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.103449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] RBAC: Authorizing `identity:get_project(project_id=52bc0b446e374669ba4bc06ffe71e784)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.104910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.107358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 62/130] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/projects/52bc0b446e374669ba4bc06ffe71e784 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:45.107786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.127618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e9112-380d-452f-98d3-9ff64e6c8c39 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.180064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.180679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.180953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.181223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.182009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=3943caa0d3d747869805423a9e86e709) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:45.182593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.184802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.201752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 69/131] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:44 2022] GET /identity/v3/role_assignments?user.id=3943caa0d3d747869805423a9e86e709&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:45.201752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.256392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:45.261084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab575-93ff-495f-b153-9cb023efb76d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.311700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.313845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.314129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.314404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.315879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.317825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49048e4a-6bcb-41e9-b113-d3691bf72cd1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.319568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 63/132] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:45 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:45.323123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d911-f72a-4fef-b5cc-bcebf5ed7f85 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.323658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d911-f72a-4fef-b5cc-bcebf5ed7f85 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.324112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d911-f72a-4fef-b5cc-bcebf5ed7f85 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.326333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 64/133] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:45.327398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90d911-f72a-4fef-b5cc-bcebf5ed7f85 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.339966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dff7b-a8d1-4dd3-a6b5-f0ec31645277 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.380855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:45.387513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.388222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.388501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.388776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.390019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.391796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.393730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 70/134] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:45 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:45.396513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.464348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90d911-f72a-4fef-b5cc-bcebf5ed7f85 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.506229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.507309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.507801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.508286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] PATH_INFO: `/v3/projects/52bc0b446e374669ba4bc06ffe71e784/users/25a810e91f074496a4fe262fab43ea53/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.510644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=52bc0b446e374669ba4bc06ffe71e784, user_id=25a810e91f074496a4fe262fab43ea53, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.513819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.537831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac876a0-efee-4cfd-9f1d-df0ed4bfa879 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.580378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.581051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.581335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.581611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.582915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.584376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc108844-c4c8-43bd-9a8b-abbde3fc05de None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:45.586260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 71/135] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:45 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:45.587235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.587523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.587804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.618033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:45.619313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:45.653286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 65/136] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:01:45 2022] PUT /identity/v3/projects/52bc0b446e374669ba4bc06ffe71e784/users/25a810e91f074496a4fe262fab43ea53/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:45.660474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.737452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:45.770537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:45.789063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 72/137] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:45.791003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:45.885631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534a02-ab10-45a1-b949-dd09d14a88b2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:45.965908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:45.966922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:45.967450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:45.967905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/65f8e87a284d46c3becdb8085954efaf/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:45.970174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, user_id=65f8e87a284d46c3becdb8085954efaf, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:45.972025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:46.004564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7ca6-4642-4425-8091-e7b80407fdec None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:46.052547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:46.053592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:46.062292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 66/138] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:01:45 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/65f8e87a284d46c3becdb8085954efaf/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:46.062940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:46.063493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/25a810e91f074496a4fe262fab43ea53/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:46.072157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, user_id=25a810e91f074496a4fe262fab43ea53, role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:46.074816 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:46.142141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:46.208162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 73/139] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:01:45 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/25a810e91f074496a4fe262fab43ea53/roles/44671b36ffe9406e9372616fdd4b9400 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:46.286144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:46.310718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314ef2a-1b69-40e5-9411-a7334448c52f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:46.368879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:46.369971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:46.370472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:46.371587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:46.373805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:46.376916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d561f0-829b-4794-9b14-19a28013613f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:46.381463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 67/140] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:46.385551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:46.467307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:46.520644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d48beb-03ec-4498-aca9-e9f8e30cae96 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:46.559192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:46.559842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:46.560130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:46.560407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:46.561148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:46.563085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affde78-322b-460d-b32f-49f8a4a72689 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:46.687897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 74/141] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:01:46 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:46.774338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61436e2-c327-4d22-80c9-7bf3fa3321bc None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:46.775277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61436e2-c327-4d22-80c9-7bf3fa3321bc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:46.775711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61436e2-c327-4d22-80c9-7bf3fa3321bc None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:46.777821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 68/142] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:46.789185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:46.789645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:46.790054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:46.818907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:46.820192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:46.923797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 75/143] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:47.420453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61436e2-c327-4d22-80c9-7bf3fa3321bc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:47.545179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61436e2-c327-4d22-80c9-7bf3fa3321bc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:47.582242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:47.583275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:47.583751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:47.584220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:47.598203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:47.603605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:47.604521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:47.605943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:47.608794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 69/144] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:47.617186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:47.672605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:47.744016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c241799-ecb2-42d9-853c-20424b65f3f2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:47.787219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:47.787847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:47.788120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:47.788388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:47.789052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:47.789346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:47.790764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:47.805480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 76/145] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:47.811122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:47.830177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:47.925247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c433a-8b57-4b8e-9d6b-d45dece4db3d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:47.963007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:47.963652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:47.963925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:47.964195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:47.966116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:47.967578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:47.969310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 70/146] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:47 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:47.978165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:48.099699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f48642-a81d-4fac-8aa7-03a7675b531f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:48.110248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:48.145592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:48.146278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:48.146555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:48.146826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:48.240291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:48.241897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f72d9-270e-47ed-a20b-ae1713e4e303 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:48.404982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 77/147] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:48.411756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:48.541930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733da99-d85a-4db6-bb45-5454490f3178 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:48.542701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733da99-d85a-4db6-bb45-5454490f3178 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:48.543291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733da99-d85a-4db6-bb45-5454490f3178 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:48.553858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 78/148] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:48.563277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:48.563897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:48.564358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:48.574729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:48.593370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d48019-f943-4269-8218-5f9a88b2effe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:48.598554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:48.624466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:48.625900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:48.636951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:48.639012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:48.639012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:48.639012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:48.640639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:48.642183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:48.644345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 71/149] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:48 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:48.650126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:48.732241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 79/150] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:48.813979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c18fe-17fe-439f-b701-9169eb759bcd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:48.864205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:48.864820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:48.865094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:48.865362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:48.866173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:48.866474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:48.867998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:48.875835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 72/151] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:48 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:49.347188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:49.491767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6efe12-b9de-4af8-ab07-19c213da89e4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:49.540281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:49.540936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.541211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.541479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.553260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-a6c9f4c9-ced3-409d-a7e7-c665131e2b57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:49.562267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:49.563200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:49.563959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:49.566649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:49.566773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 80/152] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:49.575527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:49.601802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:49.655096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70507-4385-4969-a737-f824c6e1a3fe None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.655613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70507-4385-4969-a737-f824c6e1a3fe None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.655613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea70507-4385-4969-a737-f824c6e1a3fe None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.657010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 81/153] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:49.671253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f373-29c4-48b7-9a8c-0bf869d42e95 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.671557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f373-29c4-48b7-9a8c-0bf869d42e95 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.672009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f373-29c4-48b7-9a8c-0bf869d42e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.709970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9855f373-29c4-48b7-9a8c-0bf869d42e95 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:49.711547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9855f373-29c4-48b7-9a8c-0bf869d42e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:49.740202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433f842-aa47-42be-97a1-eb1d5e1b2776 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:49.831706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 82/154] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:49.839178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:49.840580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.841081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.841555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.842377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af176f-bfdd-4ba9-bb1d-1f6dba137b51 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.842557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:49.842971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:49.843081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af176f-bfdd-4ba9-bb1d-1f6dba137b51 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.843477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af176f-bfdd-4ba9-bb1d-1f6dba137b51 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.844710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397f8ce-924f-4c94-af3c-130b6964d6a2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:49.845291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 83/155] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:49.850999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044cbc-d40f-497a-8e14-3ce7300705c0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.851533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044cbc-d40f-497a-8e14-3ce7300705c0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.852017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044cbc-d40f-497a-8e14-3ce7300705c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.868012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 73/156] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:49.869827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da829a-90f7-46d9-a500-0e85745bb170 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.870334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da829a-90f7-46d9-a500-0e85745bb170 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.871385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da829a-90f7-46d9-a500-0e85745bb170 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:49.873230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 74/157] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:49.878717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d5044cbc-d40f-497a-8e14-3ce7300705c0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:49.879237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da829a-90f7-46d9-a500-0e85745bb170 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:49.879967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d5044cbc-d40f-497a-8e14-3ce7300705c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:49.992500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 84/158] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:49.993890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:49.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:49.994441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:50.033405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:50.034627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:50.104188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:50.142107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:50.160066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da829a-90f7-46d9-a500-0e85745bb170 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:50.169339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 85/159] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:50.211814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:50.212459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:50.212734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:50.213005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] PATH_INFO: `/v3/users/3943caa0d3d747869805423a9e86e709` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:50.214234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] RBAC: Authorizing `identity:get_user(user_id=3943caa0d3d747869805423a9e86e709)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:50.216246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:50.218068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 75/160] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:49 2022] GET /identity/v3/users/3943caa0d3d747869805423a9e86e709 => generated 289 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:50.223656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:50.349421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da479920-71fb-4f35-883f-22e79b4e0248 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:50.398765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:50.399390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:50.399669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:50.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:50.401115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:50.403367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:50.405374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 86/161] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:50 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:50.411302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:50.527386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8c6b-8f47-43ac-b225-27c6884f615e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:50.551189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:50.587538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:50.588572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:50.589051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:50.589552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/3943caa0d3d747869805423a9e86e709/roles/c219c1a78e09431ab0ac1b3968aff0b5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:50.595200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, user_id=3943caa0d3d747869805423a9e86e709, role_id=c219c1a78e09431ab0ac1b3968aff0b5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:50.597084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:50.653963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:50.712825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 76/162] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:01:50 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/users/3943caa0d3d747869805423a9e86e709/roles/c219c1a78e09431ab0ac1b3968aff0b5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:01:50.714024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:50.722939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292982e7-ab09-41e4-9a0d-541e4a9013e4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:50.732363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:50.787362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:50.788075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:50.788355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:50.788619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:50.810607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:50.816387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:50.817303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:50.818113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 87/163] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:01:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:50.818756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.028720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38206f-4b29-4311-aa27-2aae5647ff41 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.045188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9d8c2-6cec-4897-9fac-c99b037afdee None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.062258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:51.106109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.106766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.107039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.107376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.129133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:51.134636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:51.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:51.137379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:51.142888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 88/164] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:51.142888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.153627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.154397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.154711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.154968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.177800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:51.187248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:51.191064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 77/165] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:51.194276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.429640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:51.467929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a5ea6-996d-4dc0-a051-5b6d8db86ef0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.531181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403f88-c6cb-4f2d-a29c-8ddba453947d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.564156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.565126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.566537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.569760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.570642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:51.571190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:51.572839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:51.589331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 89/166] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:01:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:51.591662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.602682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.609760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.610279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.610700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.612095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:51.612095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:51.613755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916869a-979f-4862-8961-fa9df1b4eb97 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:51.625425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 78/167] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:51.628838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8b38-76d0-4673-9648-ca2514cb284e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.629135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8b38-76d0-4673-9648-ca2514cb284e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.629413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f8b38-76d0-4673-9648-ca2514cb284e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.633855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 79/168] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:51.635689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8b38-76d0-4673-9648-ca2514cb284e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.669491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:51.791817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cf9b2-aa2f-4d89-b4cd-cce14fabe529 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.839721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.842087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.842538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.843495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.844788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:51.846003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:51.847796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:51.848453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f8b38-76d0-4673-9648-ca2514cb284e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:51.868917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 90/169] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:51.877411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:51.898996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:51.912246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:51.914611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.914611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.914611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.915576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:51.918319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6744bf9-d516-46a5-9db7-e6d28238a787 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:51.920632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 80/170] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:51 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:51.922205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8599cbc-6f92-45e4-a957-d6434b2de054 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:51.922571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8599cbc-6f92-45e4-a957-d6434b2de054 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:51.922844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8599cbc-6f92-45e4-a957-d6434b2de054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:51.950348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f8599cbc-6f92-45e4-a957-d6434b2de054 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:51.951592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f8599cbc-6f92-45e4-a957-d6434b2de054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:52.000039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.003801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14877180-39d1-44fd-b705-f355d8bc8574 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.036693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 81/171] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:52.044251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cab-f0a5-4a66-94f2-0bd2f8379d5d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.044251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cab-f0a5-4a66-94f2-0bd2f8379d5d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.044251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cab-f0a5-4a66-94f2-0bd2f8379d5d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.044251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 82/172] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:52.044251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71cab-f0a5-4a66-94f2-0bd2f8379d5d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:52.047697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:52.048358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.048644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.048913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] PATH_INFO: `/v3/users/25a810e91f074496a4fe262fab43ea53` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.050132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] RBAC: Authorizing `identity:get_user(user_id=25a810e91f074496a4fe262fab43ea53)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:52.051600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df038-5837-4c41-be1d-150cc5c1fdc0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:52.053217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 91/173] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:51 2022] GET /identity/v3/users/25a810e91f074496a4fe262fab43ea53 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:52.054412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b781-5912-47bd-a746-b7845a694884 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.054696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b781-5912-47bd-a746-b7845a694884 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.054967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b781-5912-47bd-a746-b7845a694884 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.056503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 92/174] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:52.057020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b781-5912-47bd-a746-b7845a694884 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:52.196958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71cab-f0a5-4a66-94f2-0bd2f8379d5d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.212857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b781-5912-47bd-a746-b7845a694884 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.233745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.251435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:52.252089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.252371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.252633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] PATH_INFO: `/v3/users/65f8e87a284d46c3becdb8085954efaf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.254532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] RBAC: Authorizing `identity:get_user(user_id=65f8e87a284d46c3becdb8085954efaf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:52.256988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6c090-4119-4f0a-89bc-8934257ddd6f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:52.258830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 83/175] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/users/65f8e87a284d46c3becdb8085954efaf => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:52.260646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:52.262038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.262693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.263454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.265100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a67e-5b5c-477e-ae20-15e87237fab6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.266263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a67e-5b5c-477e-ae20-15e87237fab6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.267124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816a67e-5b5c-477e-ae20-15e87237fab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.288780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6816a67e-5b5c-477e-ae20-15e87237fab6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:52.290264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6816a67e-5b5c-477e-ae20-15e87237fab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:52.313886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.352546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:52.354409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:52.375456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 84/176] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:52.379268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ae06-f357-408c-8b53-223337917994 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.379575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ae06-f357-408c-8b53-223337917994 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.379969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5ae06-f357-408c-8b53-223337917994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.404002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-33a5ae06-f357-408c-8b53-223337917994 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:52.405303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-33a5ae06-f357-408c-8b53-223337917994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:52.477585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 93/177] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:52.478919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:52.488969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 85/178] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:52.489861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ae06-f357-408c-8b53-223337917994 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:52.555599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.594778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f46af-1588-4b45-9c6a-0f1a8b3e1918 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.598601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.636069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:52.637164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.637164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.637403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.713995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5ae06-f357-408c-8b53-223337917994 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.719084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:52.720623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:52.753154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:52.754327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:52.754800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:52.755299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:52.791417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.831501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 94/179] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:52.837062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:52.839221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:52.841269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:52.907687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:52.973874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6eba0f-5896-4b0c-a463-b37edb553e70 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:52.979193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 86/180] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:52.979193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.030992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.031713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.031991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.032265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.045939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:53.051109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:53.052187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:53.052968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 95/181] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:53.053634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.087723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.114513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36487936-e6f7-4482-a09c-59065c623731 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.164056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.165071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.165500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.165963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.168788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.171316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.179660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 87/182] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:52 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.179660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.189846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765369aa-7469-4a87-b3e0-7c54670074c0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.208619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.231935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.232581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.232854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.233121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.234464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.235945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.237808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 96/183] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.242879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.335047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a191e-d302-4591-916d-6dde9e7dee29 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.371382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.376387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.377103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.377382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.377654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] PATH_INFO: `/v3/projects/52bc0b446e374669ba4bc06ffe71e784` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.378891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] RBAC: Authorizing `identity:get_project(project_id=52bc0b446e374669ba4bc06ffe71e784)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.379832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c2f0d-0701-4478-ba68-a904f44d7b8d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.380386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.382438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 88/184] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/projects/52bc0b446e374669ba4bc06ffe71e784 => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.383256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.444664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.445447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.445835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.446233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.458564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:53.463510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:53.464463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:53.465277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 97/185] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:53.466046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.519399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.534323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549abc0c-53f0-48e0-86f0-d00ec700341b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.575575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.576264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.576533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.576796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.577967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.579445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.581078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 89/186] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.582631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.605867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e871-fe9f-4403-8de6-46813a0231bd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.656122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.656735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.657012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.659790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.659790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:53.659790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.659790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.671486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 98/187] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.672361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.697402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.722095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259e86b-f89f-46f3-ba62-cca7d49dea6e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.759041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.759661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.760000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.760274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.761580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:53.761899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.763359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.769740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 90/188] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.770621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:53.803770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5603-c773-4108-a362-2184b6ca9384 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.806283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:53.855492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.856336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.856472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.856855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.857375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.858973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.925846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2688f-0806-4e46-95e5-f2a768f18d67 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:53.967976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:53.968605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.968879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.969207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.970013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=52bc0b446e374669ba4bc06ffe71e784, user.id=25a810e91f074496a4fe262fab43ea53) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:53.970319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:53.971826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab839331-0d05-4b5d-80fd-368a0e04cd05 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:53.983574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 91/189] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/role_assignments?user.id=25a810e91f074496a4fe262fab43ea53&scope.project.id=52bc0b446e374669ba4bc06ffe71e784&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:53.984632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72265b1a-d415-4046-8da0-772461b47979 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:53.985033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72265b1a-d415-4046-8da0-772461b47979 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:53.985354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72265b1a-d415-4046-8da0-772461b47979 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:53.986763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 92/190] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:53.987555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72265b1a-d415-4046-8da0-772461b47979 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.002864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 99/191] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:01:53 2022] POST /identity/v3/users => generated 297 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:01:54.004811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.152760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:54.166366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:54.191626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffafd3-071b-4251-b48f-6ebab3665656 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:54.205140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72265b1a-d415-4046-8da0-772461b47979 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:54.246531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:54.247109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.247390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.247656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.248714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=65f8e87a284d46c3becdb8085954efaf) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:54.249088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:54.250701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:54.265752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 100/192] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:54 2022] GET /identity/v3/role_assignments?user.id=65f8e87a284d46c3becdb8085954efaf&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:54.265752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.284260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:54.285087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.285258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.286161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.286873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:54.287247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:54.288702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddcf90-dd48-40b8-b7bb-92df177e6e1e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:54.300557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 93/193] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:01:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:54.302375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.302973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.303365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.331456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:54.332746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:54.436960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724640d5-a7c2-44be-a331-d403ae67651a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:54.451205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 94/194] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:54.453392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.512453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:54.512961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.513238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.513508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.518492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:54.533835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:54.535392 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:54.537501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 101/195] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:54 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:54.547283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.577423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:54.629733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fc7af-690b-43d4-b6cd-96ac94414309 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:54.704258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:54.705331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.705614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.705917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.707802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:54.709335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:54.711560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 95/196] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:54 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:54.734122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:54.757155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8984d-d74d-40d7-92b6-78913f87876f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:54.805174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:54.805942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:54.806223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:54.806553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:54.884802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:54.887144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:54.910138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:54.962223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c50ef8-0079-4bc0-9c29-7618cd4e49c3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.003914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 102/197] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:01:55.004967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.010037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.010987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.013378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.013896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.037098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.110158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.112701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.153563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.160953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feecbb-018c-46dc-b58b-d6bdc9945a79 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.197403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.198258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.199348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.199792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.212416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:55.219048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:55.220062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:55.221075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 103/198] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:55.222646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.248695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 96/199] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:01:55.249917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.306889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.333388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3747db5-1774-4059-93e2-06c052ecabcb None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.370749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.371492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.371875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.372252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.373847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.375427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.377613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 104/200] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:55.382803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f870d0b-5ef4-4133-bb23-853dad7e734e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.383450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.411294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.433834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.435326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.435974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.436618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.438012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:55.438722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.441400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.456917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 97/201] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:55.458765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.511395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744c25b-91a5-4f29-b75c-050b829229d4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.548631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.549430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.549723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.550120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.551406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.552947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.555047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 105/202] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:55.555618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.581016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a95c72-7e54-45c2-badb-fa0b6678bb17 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.590091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.626100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.626645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.626922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.627186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.627918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=7206c528af914219a12f9306d0c2fc3a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:55.628227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.629667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.644123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 98/203] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/role_assignments?user.id=7206c528af914219a12f9306d0c2fc3a&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:55.644123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.702905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:55.714323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048a07c-cadb-4712-b57e-5552877db34e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.771639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.772279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.772556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.772832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] PATH_INFO: `/v3/users/3943caa0d3d747869805423a9e86e709` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.781756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] RBAC: Authorizing `identity:get_user(user_id=3943caa0d3d747869805423a9e86e709)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.781756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.781756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 106/204] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/users/3943caa0d3d747869805423a9e86e709 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:55.783531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:55.864653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6b114-121c-47b2-ab6e-d669950c46ab None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:55.910787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:55.912124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:55.912802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:55.913500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:55.915635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=7206c528af914219a12f9306d0c2fc3a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:55.916642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:55.920447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:55.930821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 99/205] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/role_assignments?user.id=7206c528af914219a12f9306d0c2fc3a&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:56.025877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8d065-fbec-49fa-a088-f951c3afbf8f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:56.076262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:56.076956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.077237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.077504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.118856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:56.206717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:56.208135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:56.406517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 107/206] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:56.412500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:56.491826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:56.598983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ece29-3753-406a-ade3-a3c5dda3743e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:56.665229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:56.666346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.666822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.667845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.669382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:56.671174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65986e1-c44e-422c-8209-ea8a55c9b84e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:56.673402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 100/207] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:56 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:56.679135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:56.752062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01085484-902b-4597-8c9e-00ec06c6dc7e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.752354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01085484-902b-4597-8c9e-00ec06c6dc7e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.752626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01085484-902b-4597-8c9e-00ec06c6dc7e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.755030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 101/208] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:01:56.762565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b97cf2-e525-4f7a-92e5-bfa2aaed71f0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.762855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b97cf2-e525-4f7a-92e5-bfa2aaed71f0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.763126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b97cf2-e525-4f7a-92e5-bfa2aaed71f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.793136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:56.798887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-67b97cf2-e525-4f7a-92e5-bfa2aaed71f0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:56.800112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-67b97cf2-e525-4f7a-92e5-bfa2aaed71f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:56.881268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255acd03-bc41-410e-89e1-2306345d51a5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:56.935783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:56.937139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.937862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.938563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.940898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, scope.project.id=90460bb6976b4887b7303a9cbeec4b38, user.id=3943caa0d3d747869805423a9e86e709) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:56.941700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:56.944659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01157e-1c43-4b5f-bad9-0d0f4ab0ac5f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:56.958506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 108/209] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:56 2022] GET /identity/v3/role_assignments?user.id=3943caa0d3d747869805423a9e86e709&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:56.958885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 102/210] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:56.960401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498fdcc-ca18-47b6-a0ce-db5d3bcf4603 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.960900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498fdcc-ca18-47b6-a0ce-db5d3bcf4603 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.961375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498fdcc-ca18-47b6-a0ce-db5d3bcf4603 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:56.963461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 109/211] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:56.970650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:56.970650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:56.970650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:57.003592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:57.004704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:57.049755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:57.116483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 103/212] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:57.419798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498fdcc-ca18-47b6-a0ce-db5d3bcf4603 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:57.463618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:57.610164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498fdcc-ca18-47b6-a0ce-db5d3bcf4603 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:57.637754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:57.650754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:57.651528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:57.651803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:57.652070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:57.654062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28413b90-e6b4-4f24-bfb8-73c142f9ac2d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:57.663827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:57.670547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:57.672523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:01:57.682442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 110/213] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:57.689562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:57.716789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:57.717923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:57.718422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:57.718951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:57.720320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:57.722084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:57.724545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 104/214] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:57.731025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:57.783678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:57.828145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30496e-2844-492a-a3e1-d15db26d0802 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:57.879587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:57.880930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:57.881390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:57.881874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:57.882742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:57.882849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a48b1-7589-4885-a35a-7ddcd098d3cf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:57.883339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:57.885296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:57.897411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 111/215] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:01:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:57.904345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:57.944631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:57.945366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:57.945787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:57.946194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:57.947288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:57.949498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:57.978472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:58.061841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ceb830-529a-4450-87d7-9d04d57873b0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:58.106690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:58.107456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.107736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.108008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] PATH_INFO: `/v3/users/d3df38e65f7c4df68c3df6779a3d0afa` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.109605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 105/216] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:01:57 2022] POST /identity/v3/users => generated 297 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:01:58.127542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] RBAC: Authorizing `identity:get_user(user_id=d3df38e65f7c4df68c3df6779a3d0afa)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:58.129822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1530940-63ef-4dd6-8613-5a12e8215655 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:58.131765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 112/217] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:57 2022] GET /identity/v3/users/d3df38e65f7c4df68c3df6779a3d0afa => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:58.140520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:58.252681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:58.271252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561b78b-1c27-4318-b62f-e5ba520b56c8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:58.310041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:58.310674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.310955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.311228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.322413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4f7d-b8b8-4be4-8cfa-4389709cf9f7 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.323061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4f7d-b8b8-4be4-8cfa-4389709cf9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.324186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4f7d-b8b8-4be4-8cfa-4389709cf9f7 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.326402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 113/218] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:58.332539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e30-7709-4065-ae55-1de82cd1cbe1 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.332957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e30-7709-4065-ae55-1de82cd1cbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.333323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2e30-7709-4065-ae55-1de82cd1cbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.355879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f47f2e30-7709-4065-ae55-1de82cd1cbe1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:58.357058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f47f2e30-7709-4065-ae55-1de82cd1cbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:58.416827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:58.419738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:58.455659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 114/219] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:01:58.457737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113f24-2c4b-4116-8294-4047a20ba68f None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.458232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113f24-2c4b-4116-8294-4047a20ba68f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.458712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25113f24-2c4b-4116-8294-4047a20ba68f None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.461539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 115/220] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:01:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:01:58.469960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.470364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.470757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.497965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:01:58.499305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:01:58.535407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:58.602990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:58.622408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 116/221] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:01:58.644918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 106/222] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:01:58.654438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:58.872217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f80b6c-97b4-42c7-aab7-ea6755482c12 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:58.918370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:58.949915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:58.950990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:58.951531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:58.952037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:58.955556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:58.957370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:58.959588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 117/223] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:58 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:58.965751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.025706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:59.101219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269947f-bce7-4631-a9e6-3c0e3938e0cb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.167782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.168354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.168625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.168890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.183044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825b2d3-ce08-4cfa-b9fe-b5d49439c2af None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.188085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:59.193275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:59.194219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:01:59.195009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:59.203082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 107/224] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:01:59.204848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.245643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.247216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.248029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.248503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.249887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=d3df38e65f7c4df68c3df6779a3d0afa) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:59.250446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:59.252158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:59.266178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 118/225] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:01:58 2022] GET /identity/v3/role_assignments?user.id=d3df38e65f7c4df68c3df6779a3d0afa&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:01:59.267633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.307249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:59.427662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419755-7050-4e65-9741-9b19feeee1f4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.500961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.501787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:59.502228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.502504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.503761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.518363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cf8db-73b7-45f5-b0ba-903da323b030 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.529341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:01:59.543641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:01:59.544619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:01:59.545578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:01:59.546280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 108/226] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:01:59.554768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.590088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.591036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.591459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.591863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.592882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:59.593337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:59.597340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:59.633815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 119/227] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:59.640522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.706011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b12f2-878e-4e9b-9396-ba03e60ed7d4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.715926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:01:59.754426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.755333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.755761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.756247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.757203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:01:59.757665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:59.759317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:59.769119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e02ae-c677-4ea6-9434-b7f62e02ef0d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:01:59.784698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 109/228] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:59.790821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.831721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:01:59.832383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:01:59.832658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:01:59.832927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:01:59.838571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:01:59.840161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:01:59.842182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 120/229] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:01:59.847769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:01:59.879671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:00.025437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63266e-e044-4d1e-990f-dad277d65229 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:00.044129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab655609-e40c-4e6b-b9b3-79f1ea3ae404 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:00.125310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:00.126406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:00.126884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:00.127359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:00.132749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:00.135370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:00.143855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 121/230] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:00.149981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:00.151893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:00.152370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:00.152853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:00.154986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:00.157048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:00.159854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 110/231] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:01:59 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:00.164878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:00.174659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:00.289165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:00.357997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:00.374999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd671e-49a4-41c1-84c9-198547e46a98 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:00.452180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:00.454168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:00.456481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:00.457007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/users/7206c528af914219a12f9306d0c2fc3a/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:00.460953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, user_id=7206c528af914219a12f9306d0c2fc3a, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:00.464573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988d175-d3c3-4a52-9288-bc9831fd1bb7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:00.482419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6e2e3-d43c-47cb-8724-7ca72aab0cbe None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:00.550710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:00.551721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:00.552222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:00.552776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:00.554971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:00.558331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:00.561612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 111/232] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:00 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:00.571435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:00.596482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 122/233] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:00 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/users/7206c528af914219a12f9306d0c2fc3a/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:00.784029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436d796-856b-4777-9d58-080b38454f7b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:00.791186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:00.856845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:00.857908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:00.858410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:00.858907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/users/7206c528af914219a12f9306d0c2fc3a/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:00.863114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, user_id=7206c528af914219a12f9306d0c2fc3a, role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:00.865910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6550c-5a28-456e-b589-fdfd0b655839 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:01.005920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 112/234] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:00 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/users/7206c528af914219a12f9306d0c2fc3a/roles/44671b36ffe9406e9372616fdd4b9400 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:01.084821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21255bf-5a52-4f8b-b0f0-5cb4c5297f21 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:01.085265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21255bf-5a52-4f8b-b0f0-5cb4c5297f21 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:01.085416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21255bf-5a52-4f8b-b0f0-5cb4c5297f21 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:01.087128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 123/235] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:01.093661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220b47-ee22-4971-a34f-cad43efdd884 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:01.094086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220b47-ee22-4971-a34f-cad43efdd884 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:01.094235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220b47-ee22-4971-a34f-cad43efdd884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:01.118717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-13220b47-ee22-4971-a34f-cad43efdd884 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:01.119844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-13220b47-ee22-4971-a34f-cad43efdd884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:01.224596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 113/236] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:01.448147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171af285-1ff6-46f1-9a0e-9f4760f312fc None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:01.448370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171af285-1ff6-46f1-9a0e-9f4760f312fc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:01.448773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171af285-1ff6-46f1-9a0e-9f4760f312fc None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:01.450236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 124/237] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:01.455837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:01.456194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:01.456431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:01.477151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:01.478369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:01.546056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171af285-1ff6-46f1-9a0e-9f4760f312fc None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:01.593547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 114/238] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:01.732984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171af285-1ff6-46f1-9a0e-9f4760f312fc None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:01.772624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:01.773277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:01.773550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:01.773848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:01.786596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:01.793544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:01.795232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:01.796649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 125/239] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:01.803572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:01.869041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:02.056498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56294c28-af55-4cb6-a8e7-3e852d2fad70 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:02.137110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:02.138144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:02.138625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:02.139107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:02.140335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:02.140896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:02.143647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:02.168089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 115/240] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:02.174222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:02.177846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:02.203487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:02.353371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d674-6986-4174-8c62-cc39883e10b4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:02.385642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:02.396740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:02.397738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:02.398164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:02.398638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] PATH_INFO: `/v3/users/69f648569d6d4362b1ec54fb90b91927` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:02.414656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=69f648569d6d4362b1ec54fb90b91927)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:02.416454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:02.421819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611691f9-bdab-4a9b-8441-77776105a37c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:02.423426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 126/241] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:02 2022] GET /identity/v3/users/69f648569d6d4362b1ec54fb90b91927 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:02.424784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:02.505533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:02.529370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:02.530054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:02.530336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:02.530600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:02.538048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:02.539695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:02.540109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d7824-c6a2-4d6c-b371-cf2e61a45ac6 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:02.541695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 116/242] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:02.549575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:02.664427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:02.665878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:02.666571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:02.667798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:02.852605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db810f13-065f-400a-ab54-a1ab0cfde569 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:02.893591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:02.897759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:02.897759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:02.897759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:02.897759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:02.897759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d4696-df1a-4b59-bb61-5e660a610239 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:02.934721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:02.937669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:03.002096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 117/243] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:02 2022] POST /identity/v3/users => generated 293 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:02:03.008045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764a0c-c891-499f-bf6b-9287ff2e34ec None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.008543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764a0c-c891-499f-bf6b-9287ff2e34ec None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.012082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764a0c-c891-499f-bf6b-9287ff2e34ec None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.016259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 118/244] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:03.024685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.025216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.025700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.100663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:03.103093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:03.115556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:03.180412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:03.272026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 127/245] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:02:03.284203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:03.362820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 119/246] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:03.508102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38197e19-e8f4-4f96-b94c-52e5f310feb0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:03.581677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:03.582771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.583258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.583753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.585910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:03.589443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e29726-d8dc-4d8c-8838-1f1a0cfdc677 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:03.593772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 128/247] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:03 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:03.603694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:03.641599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089690b7-c125-47d7-8f1a-197f058a575e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.642139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089690b7-c125-47d7-8f1a-197f058a575e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.642712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089690b7-c125-47d7-8f1a-197f058a575e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.645430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 129/248] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:03.656518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.656518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.656925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.688333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:03.692957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:03.755887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573ec10-dde8-4ef3-b74b-2a229b2a73ef None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:03.760313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:03.821692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:03.822703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.823665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.823914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.824931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=69f648569d6d4362b1ec54fb90b91927) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:03.825186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:03.826687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aed3fc-ff95-42c3-9407-95173cf80268 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:03.827106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 130/249] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:03.828850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:03.834047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 120/250] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:03 2022] GET /identity/v3/role_assignments?user.id=69f648569d6d4362b1ec54fb90b91927&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:03.857655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d52c4-ea51-4cb3-b9e9-85b23544d060 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.858570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d52c4-ea51-4cb3-b9e9-85b23544d060 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.858570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d52c4-ea51-4cb3-b9e9-85b23544d060 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.860157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 121/251] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:03.867586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:03.868153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:03.868494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:03.884277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:03.910406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:03.911396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:04.054862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3c93d-9274-451c-843a-5707fa990f7b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:04.119326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 122/252] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:04.132953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:04.134369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:04.135078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:04.136280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:04.156914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:04.167449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:04.169229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:04.170695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:04.175012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 131/253] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:04.185745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:04.336562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:04.435191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:04.471646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93dba3-f1f6-4d0d-8665-1588978fbe5b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:04.574315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:04.574941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:04.575214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:04.575473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:04.576255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:04.576556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:04.578005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:04.582224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:04.600089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 123/254] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:04.611550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:04.655794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0968d56-54e6-4183-850f-8398787a61df None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:04.778547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:04.779210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:04.779516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:04.779753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:04.818876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:04.824085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:04.826232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:04.828074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:04.833844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 132/255] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:04.834995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:04.877375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79babc9b-3f25-4b17-9c99-c8dc943bff23 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:04.943143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:04.944262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:04.944758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:04.945235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] PATH_INFO: `/v3/users/d3df38e65f7c4df68c3df6779a3d0afa` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:04.947425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] RBAC: Authorizing `identity:get_user(user_id=d3df38e65f7c4df68c3df6779a3d0afa)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:04.952731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:04.955103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 124/256] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:04 2022] GET /identity/v3/users/d3df38e65f7c4df68c3df6779a3d0afa => generated 297 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:04.955900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:05.098357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:05.108965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:05.129471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d86bd-24bc-402b-b2a5-8ceaef6d5c5e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:05.223218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:05.224959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:05.225828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:05.226534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:05.247839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:05.260400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:05.262447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:05.264087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:05.266785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426e1c4-c2e9-4270-a3dd-ece9f7152255 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:05.269842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 133/257] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:02:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:05.271707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:05.410184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:05.410718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:05.410996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:05.411284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:05.411938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:05.412234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:05.413622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:05.430701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 125/258] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:05.431563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:05.468552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7cf34-f633-43bd-ab8a-6233bc222e4c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:05.531418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:05.532952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:05.533684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:05.534516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:05.536824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:05.539921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:05.544102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 134/259] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:05 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:05.550519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:05.587813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:05.711684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767f85e-7172-454b-9094-c8fa183ac2e2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:05.774066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:05.803895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:05.804900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:05.805327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:05.805823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:05.811175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:05.813930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:05.816481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:05.839835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14d409-69a2-4f42-9588-8f8544798b29 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:05.856878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 126/260] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:05.857898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:05.920093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:05.920743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:05.921022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:05.921393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:05.923266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:05.929948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:05.932272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 135/261] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:05 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:05.934192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:06.085926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d06053-1fef-4cf6-ace7-1d326de8ab63 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:06.096016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:06.127701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33760-4d96-4605-9544-29d9b8bdda79 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:06.155364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:06.156188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:06.156462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:06.156733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/users/d3df38e65f7c4df68c3df6779a3d0afa/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:06.159976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, user_id=d3df38e65f7c4df68c3df6779a3d0afa, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:06.161432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:06.187010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:06.187710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:06.187993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:06.188267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] PATH_INFO: `/v3/users/7206c528af914219a12f9306d0c2fc3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:06.189546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] RBAC: Authorizing `identity:get_user(user_id=7206c528af914219a12f9306d0c2fc3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:06.191151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:06.192848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 136/262] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:05 2022] GET /identity/v3/users/7206c528af914219a12f9306d0c2fc3a => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:06.193918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:06.253653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 127/263] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:05 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/users/d3df38e65f7c4df68c3df6779a3d0afa/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:06.254646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:06.382367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:06.392293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce24c7e-c266-4959-bee9-8ee4f7990cca None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:06.481793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:06.482447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:06.482721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:06.482986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:06.490163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787c20-d8e9-4e83-a7a4-ab7bd7b76c26 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:06.544634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:06.596064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:06.596685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:06.596960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:06.597225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:06.657102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:06.660822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c7799-43cf-4931-ad5b-a5d013891fac None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:06.786398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:06.787890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dead92c-7ba3-447c-aed1-07e8da22d616 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:06.872904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:06.901979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 137/264] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:06.904562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a05c-1cc9-47a9-963f-74e59232e364 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:06.905259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a05c-1cc9-47a9-963f-74e59232e364 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:06.905762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438a05c-1cc9-47a9-963f-74e59232e364 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:06.907828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 138/265] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:06.914022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a05c-1cc9-47a9-963f-74e59232e364 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:06.987388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:07.040839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 128/266] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 787 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:07.042667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.043000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.043292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.087741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:07.090264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:07.123427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438a05c-1cc9-47a9-963f-74e59232e364 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:07.168101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 129/267] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:07.168928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:07.202079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:07.202899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.203039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.203357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.205079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:07.206893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c4db0-b907-4ecd-95a6-f27b90aecdc4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:07.208983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 139/268] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:06 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:07.215608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83658be-01fe-4394-b6f1-e72efdc23059 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.216031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83658be-01fe-4394-b6f1-e72efdc23059 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.216834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83658be-01fe-4394-b6f1-e72efdc23059 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.221533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 140/269] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:07.225469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83658be-01fe-4394-b6f1-e72efdc23059 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:07.393657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:07.430668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60833f1-0857-4eb9-a317-ee68dfb3cd0e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:07.485887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83658be-01fe-4394-b6f1-e72efdc23059 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:07.502240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:07.503352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.503855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.504357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.506593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:07.509527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a8d08-8b74-48ad-9d3b-4b1ff881c1d1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:07.518894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:07.520522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 130/270] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:07 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:07.523007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.523503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.524001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.540070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:07.540707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.540978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.541246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.543018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=7206c528af914219a12f9306d0c2fc3a) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:07.543432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:07.544982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d8b4-fe2a-445a-99fa-99d6a67ece0f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:07.553182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 141/271] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:07 2022] GET /identity/v3/role_assignments?user.id=7206c528af914219a12f9306d0c2fc3a&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:07.555624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52020896-b5fd-438f-9594-387271a467d5 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.556037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52020896-b5fd-438f-9594-387271a467d5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.556190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52020896-b5fd-438f-9594-387271a467d5 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.558379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 142/272] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:07.559641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52020896-b5fd-438f-9594-387271a467d5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:07.569575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:07.569575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:07.737742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 131/273] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:07.737742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:07.754797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52020896-b5fd-438f-9594-387271a467d5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:07.804882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:07.805596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.805898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.806330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.808306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:07.811078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cd2a9-9683-4549-9d29-362e1cd73a5e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:07.813192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 143/274] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:07.815307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.815588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.815857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.842129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:07.844495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:07.846600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:07.867090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650d8d-7810-490f-94f0-f7157f35d0c9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:07.923011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:07.923635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:07.923909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:07.924175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:07.925006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=7206c528af914219a12f9306d0c2fc3a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:07.925312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:07.927717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:07.932222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:07.935844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 132/275] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:07 2022] GET /identity/v3/role_assignments?user.id=7206c528af914219a12f9306d0c2fc3a&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:07.936773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:07.961395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 144/276] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:08.074359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.119315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bba05-a53a-49b5-bc66-d1f05d9781a9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.190457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.191099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.191374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.191647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.192331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:08.193900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:08.214770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:08.294733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b82da3-a893-4c52-becf-9702fed2659d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.347063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 133/277] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:07 2022] POST /identity/v3/users => generated 293 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:08.347063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.357043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.358165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.358693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.359223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.381202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:08.392284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:08.394028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:08.395939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 145/278] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:08.404327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.471627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:08.492545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8c04f-f548-483d-bacc-c6eef9309a93 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.539436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076ae72-8983-4612-8784-4a0c775c8242 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.543918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.543918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.543918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.543918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.561952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:08.568534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:08.574550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:08.576153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 134/279] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:08.583508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.621981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.623426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.623975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.624603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.628814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:08.628814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:08.631507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:08.658533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 146/280] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:08.659823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.686511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:08.755206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf12ab-9aea-49be-b9e3-76554491bee8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.811224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143d978-003d-41c3-9572-cd653681affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.812159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143d978-003d-41c3-9572-cd653681affc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.812525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143d978-003d-41c3-9572-cd653681affc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.812906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143d978-003d-41c3-9572-cd653681affc None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.814126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143d978-003d-41c3-9572-cd653681affc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:08.814657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143d978-003d-41c3-9572-cd653681affc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:08.817002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143d978-003d-41c3-9572-cd653681affc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:08.835139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 135/281] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:08.849402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143d978-003d-41c3-9572-cd653681affc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:08.850935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ff68-eed0-4005-a4c0-df5a4f3fa420 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:08.858681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:08.934735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:08.938115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:08.938934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:08.939624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] PATH_INFO: `/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:08.962068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] RBAC: Authorizing `identity:get_user(user_id=b5fa74ca55f64396aa6c7f08c53be9e7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:08.966080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:08.968341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 147/282] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7 => generated 293 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:08.974384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:09.020689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143d978-003d-41c3-9572-cd653681affc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:09.059022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:09.105582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:09.106296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.106574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.106841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] PATH_INFO: `/v3/users/69f648569d6d4362b1ec54fb90b91927` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.108036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445b602-fcc5-49a0-9413-faed1d024d90 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:09.110437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] RBAC: Authorizing `identity:get_user(user_id=69f648569d6d4362b1ec54fb90b91927)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:09.111903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:09.117739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 136/283] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/users/69f648569d6d4362b1ec54fb90b91927 => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:09.131338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:09.158045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:09.159027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.159526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.160478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.161701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, user.id=b5fa74ca55f64396aa6c7f08c53be9e7, scope.system=all) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:09.162259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:09.164699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2729775-f76b-4b97-a3c2-0815f27801db None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:09.173407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 148/284] 10.222.0.99 () {56 vars in 1308 bytes} [Fri Sep 2 21:02:08 2022] GET /identity/v3/role_assignments?user.id=b5fa74ca55f64396aa6c7f08c53be9e7&scope.system=all&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:09.179257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68fffd-2410-4da1-b1c2-6c9bdbcab8a2 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.179546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68fffd-2410-4da1-b1c2-6c9bdbcab8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.179798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68fffd-2410-4da1-b1c2-6c9bdbcab8a2 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.182012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 149/285] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:09.189873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.190464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.190935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.220300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:09.221868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:09.253875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:09.292994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb515-b0c1-4d6e-b9c0-5de26afd708f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:09.318639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 150/286] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:09.354612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:09.355457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.355715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.355981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.359655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:09.361256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:09.363610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 137/287] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:09 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:09.371393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:09.524246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaedb01-6355-43d0-ab6d-036ff6825da5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:09.577892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:09.578955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.579363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.579769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/users/69f648569d6d4362b1ec54fb90b91927/roles/c219c1a78e09431ab0ac1b3968aff0b5` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.582987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, user_id=69f648569d6d4362b1ec54fb90b91927, role_id=c219c1a78e09431ab0ac1b3968aff0b5)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:09.585244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:09.639995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:09.694522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 151/288] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:09 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/users/69f648569d6d4362b1ec54fb90b91927/roles/c219c1a78e09431ab0ac1b3968aff0b5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:09.720776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:09.782737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb5088-bccf-4b8c-9c5b-99be4cb44d89 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:09.834832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:09.836027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:09.836344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:09.836737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:09.849746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:09.856260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:09.857246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:09.858069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 138/289] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:09.864055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:09.886913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:09.988401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a370c-9949-458c-989c-4ec49e9aa566 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:10.029327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:10.029977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:10.030294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:10.030568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:10.031244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:10.031539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:10.033028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:10.043751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 152/290] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:10.049553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:10.194661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24b8aa-30bb-47e4-8629-022645148207 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:10.219093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:10.246774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:10.247449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:10.247977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:10.248335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] PATH_INFO: `/v3/users/d3df38e65f7c4df68c3df6779a3d0afa` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:10.249587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] RBAC: Authorizing `identity:get_user(user_id=d3df38e65f7c4df68c3df6779a3d0afa)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:10.251101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:10.257754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 139/291] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:10 2022] GET /identity/v3/users/d3df38e65f7c4df68c3df6779a3d0afa => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:10.258702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:10.388352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:10.422662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1be8f6-b113-4442-9413-3b510730719d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:10.474850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:10.475483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:10.475755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:10.476026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:10.585480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:10.587001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:10.721379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:10.750453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 153/292] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:02:10.756201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:10.898376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567f6e-d3ab-4202-9f2a-5402e69f3ceb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:10.947184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:10.947184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:10.947496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:10.948163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:10.950423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:10.951891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301607e-bd3d-4ea8-81dd-3dfb81cb864b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:10.953548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 140/293] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:10 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:10.961459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:11.103316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8e6c3-0862-428f-b5cf-f2efaf16c214 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:11.156412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:11.157043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.157319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.157581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.158375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=d3df38e65f7c4df68c3df6779a3d0afa) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:11.158678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:11.160159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8422-a33a-4bb7-a867-4771331ef05c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:11.166991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 154/294] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:10 2022] GET /identity/v3/role_assignments?user.id=d3df38e65f7c4df68c3df6779a3d0afa&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:11.350170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2d09-7de7-4d7a-a9b8-19e5f09e2d1e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.350698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2d09-7de7-4d7a-a9b8-19e5f09e2d1e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.350933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2d09-7de7-4d7a-a9b8-19e5f09e2d1e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.353122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 141/295] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:11.359755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.360131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.360570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.386509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4c55-44fd-4a4b-819a-69465c67fa2e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.386839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4c55-44fd-4a4b-819a-69465c67fa2e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.387064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb4c55-44fd-4a4b-819a-69465c67fa2e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.388441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 142/296] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:11.394064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517088-11e0-4d3d-838b-cebba299b297 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.394468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517088-11e0-4d3d-838b-cebba299b297 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.394849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46517088-11e0-4d3d-838b-cebba299b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.395402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:11.397143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:11.417724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-46517088-11e0-4d3d-838b-cebba299b297 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:11.419121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46517088-11e0-4d3d-838b-cebba299b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:11.455485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:11.476707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:11.493536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 155/297] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:11.505835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 143/298] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:11.816019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:11.851981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce742-9942-4780-8762-bcad318bd8fc None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.852323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce742-9942-4780-8762-bcad318bd8fc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.852540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ce742-9942-4780-8762-bcad318bd8fc None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.854218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 144/299] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:11.860094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.860429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:11.860632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:11.882604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:11.883871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:11.937138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74004c67-007b-4a07-80b8-61b9622c7332 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:11.963852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 145/300] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:11.965587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:11.998043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:11.999216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:11.999783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.000339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.019639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:12.030001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:12.031865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:12.033480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:12.037500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 156/301] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:12.047713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.105366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:12.158416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590737e-27f9-4ccf-ab01-5b32a6091943 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.178916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:12.218527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.220007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.220745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.221441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.223535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:12.226444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:12.230497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 146/302] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:12.237886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc001-f5de-4acd-ac00-28fc640926da None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.240131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.285527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.286344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.286794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.287239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.288035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:12.288596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:12.290598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:12.305762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 157/303] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:12.315273 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.393883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254c486-a563-482b-8f78-d1c37d2e176c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.433301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:12.435236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.436060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.436402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.436735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] PATH_INFO: `/v3/groups` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.437543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:12.439576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d2a46-8489-477a-9f7f-e4887550cf04 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:12.440918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861d8c5-8cda-474c-b8ca-f1153516dd9b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.475922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.476562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.476838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.477107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.489183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:12.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:12.501389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 158/304] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:12.502264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.586309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 147/305] 10.222.0.99 () {60 vars in 1134 bytes} [Fri Sep 2 21:02:12 2022] POST /identity/v3/groups => generated 229 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:02:12.588709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d385f-186c-44bc-900d-9f0e43e19388 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.589193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d385f-186c-44bc-900d-9f0e43e19388 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.589657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d385f-186c-44bc-900d-9f0e43e19388 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.592150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 148/306] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:12.593472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d385f-186c-44bc-900d-9f0e43e19388 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.622049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517626b-d68b-40ef-96d5-ba72872d04a9 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.672401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:12.685703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.686542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.686986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.691165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] PATH_INFO: `/v3/users/14c102ffacb54cfa8393c88fda6b7961` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.709423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] RBAC: Authorizing `identity:get_user(user_id=14c102ffacb54cfa8393c88fda6b7961)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:12.711423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203d624-2806-402e-bcac-cd8195a01570 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:12.715145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 159/307] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/users/14c102ffacb54cfa8393c88fda6b7961 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:12.716763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.717228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.717592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.743133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:12.745849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:12.760781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:12.783640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d385f-186c-44bc-900d-9f0e43e19388 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:12.835436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:12.837031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:12.837532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:12.838814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:12.840234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:12.840930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:12.848110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:12.852608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 149/308] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:12.853401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:12.866467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 160/309] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:12.867360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:13.001271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:13.006568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a5223-8238-4f22-b020-991d772b85d4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.052804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.053735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.054140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.054501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.055351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, user.id=14c102ffacb54cfa8393c88fda6b7961, scope.system=all) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:13.055856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:13.058502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:13.064850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72348-d55e-492c-8e49-8e44d0b13b5c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.069244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 150/310] 10.222.0.99 () {56 vars in 1308 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/role_assignments?user.id=14c102ffacb54cfa8393c88fda6b7961&scope.system=all&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:13.139935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.141682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.142563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.143442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] PATH_INFO: `/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.146116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] RBAC: Authorizing `identity:get_user(user_id=b5fa74ca55f64396aa6c7f08c53be9e7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:13.149265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:13.153093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 161/311] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:12 2022] GET /identity/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7 => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:13.163116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:13.178853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:13.194298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:13.319397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f9ba0-14cc-44a9-b581-f5a7e540b3cf None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.362116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8d5b-ea9e-4ab2-b01f-c04541d1f2c7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.367048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.368767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.368767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.368767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.380884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:13.386713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:13.388049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:13.389366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:13.392626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 162/312] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:13.394057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:13.421916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.424002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.424816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.425559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] PATH_INFO: `/v3/system/users/b5fa74ca55f64396aa6c7f08c53be9e7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.428580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5fa74ca55f64396aa6c7f08c53be9e7, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:13.431593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:13.482864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:13.535214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc0dc7-db71-47ce-9211-307ccc7b5384 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.560865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 151/313] 10.222.0.99 () {58 vars in 1334 bytes} [Fri Sep 2 21:02:13 2022] PUT /identity/v3/system/users/b5fa74ca55f64396aa6c7f08c53be9e7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:13.595069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.596990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.598016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.598904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.600582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:13.601557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:13.606900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:13.633887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 163/314] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:13.639971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:13.704595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:13.837426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a04eb-c421-40a6-9df8-762b29c42be9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:13.906002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:13.906922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:13.907401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:13.907879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] PATH_INFO: `/v3/users/69f648569d6d4362b1ec54fb90b91927` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:13.909887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] RBAC: Authorizing `identity:get_user(user_id=69f648569d6d4362b1ec54fb90b91927)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:13.912498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:13.915990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 152/315] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:13 2022] GET /identity/v3/users/69f648569d6d4362b1ec54fb90b91927 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:13.922769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:14.131735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf078-1ef4-43d3-8e30-9b0e44865930 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:14.186792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:14.195370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:14.196506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:14.201425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:14.201425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:14.313755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:14.318767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:14.449560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 164/316] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:14.458986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:14.639291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:14.649859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766bea5-59d9-4d1e-a335-dd4d7cac39df None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:14.713953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:14.715047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:14.715634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:14.716133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:14.718248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:14.721086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edcd39-e859-4734-aeda-fb71ddf9a252 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:14.730161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 153/317] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:14 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:14.735351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:14.760252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3bb33-9cf4-4568-b77f-b239010b8277 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:14.760521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3bb33-9cf4-4568-b77f-b239010b8277 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:14.760879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3bb33-9cf4-4568-b77f-b239010b8277 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:14.763262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 154/318] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:14.769973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:14.770510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:14.770933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:14.795420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:14.795785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:14.797376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:14.894100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c9bf-c092-4d24-bd0b-bf48a00d7ec4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:14.903847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 155/319] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:14.934722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:14.935609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:14.936043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:14.936457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:14.937316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, scope.project.id=f54be7acf91e47c5bd233f1914534020, user.id=69f648569d6d4362b1ec54fb90b91927) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:14.937782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:14.939340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:14.946014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 165/320] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:14 2022] GET /identity/v3/role_assignments?user.id=69f648569d6d4362b1ec54fb90b91927&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:15.180229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:15.303024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:15.312202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ddbad-0bcf-4848-89b4-afd6b6dfc1aa None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:15.351192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:15.351839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.352116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.352390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.353507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:15.355590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795438b3-3c43-4b81-8e7e-286dedf6d54d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:15.357437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 156/321] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:15.363275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:15.448100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422942dc-3c8c-4f93-a80c-36373c3dbf55 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.448333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422942dc-3c8c-4f93-a80c-36373c3dbf55 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.448628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422942dc-3c8c-4f93-a80c-36373c3dbf55 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.450087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 157/322] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:15.455312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.455607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.455860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.473635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:15.475754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:15.476949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:15.480206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef358a0-78d3-42a3-8954-d359eaddbc55 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:15.518167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:15.518960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.519238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.519504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] PATH_INFO: `/v3/groups` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.520131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:15.521609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70187f69-f7d8-4246-962b-4c7561458ecc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:15.577766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 158/323] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:15.690107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 166/324] 10.222.0.99 () {60 vars in 1134 bytes} [Fri Sep 2 21:02:15 2022] POST /identity/v3/groups => generated 226 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:15.862966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:15.889597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193da32-e8b0-4548-a900-df75de9202ef None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.890154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193da32-e8b0-4548-a900-df75de9202ef None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.890154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193da32-e8b0-4548-a900-df75de9202ef None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.892585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 167/325] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:15.897311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:15.897653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:15.897964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:15.928984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:15.930311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:16.010185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:16.023027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f222c-f4da-402f-9d27-8011b6b02b40 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.036608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 168/326] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:16.087886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.089555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.090048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.090445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.106744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:16.113792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:16.115530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:16.116395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 159/327] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:16.125670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:16.295696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:16.323529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80c8a5-1fea-4b3f-9fe1-5c1d383022ed None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.384602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.386149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.386640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.387112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.388298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:16.388843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:16.391324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:16.409582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 169/328] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:16.417508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:16.559175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:16.580948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb5eb-7e64-4318-90eb-17b70e9d7684 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.648414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.649543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.650101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.651113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] PATH_INFO: `/v3/users/14c102ffacb54cfa8393c88fda6b7961` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.653074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] RBAC: Authorizing `identity:get_user(user_id=14c102ffacb54cfa8393c88fda6b7961)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:16.654966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:16.657669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 160/329] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:16 2022] GET /identity/v3/users/14c102ffacb54cfa8393c88fda6b7961 => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:16.670844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:16.711958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e57597-3282-49f0-be93-49db443fa395 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.714288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:16.758049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.759550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.760624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.761385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.775342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:16.781980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:16.782880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:16.783661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:16.786376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 170/330] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:16.797902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:16.811924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:16.885844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ed3e8-2901-468b-bc83-6bae13f34e9b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.923113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff2d-1e6b-4010-8122-ece1d1a431d7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:16.949950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.951408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.951408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.951708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] PATH_INFO: `/v3/system/users/14c102ffacb54cfa8393c88fda6b7961/roles/c219c1a78e09431ab0ac1b3968aff0b5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.953504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=14c102ffacb54cfa8393c88fda6b7961, role_id=c219c1a78e09431ab0ac1b3968aff0b5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:16.955535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:16.972905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:16.975006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:16.975006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:16.975006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:16.975006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:16.975287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:16.976550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:16.988811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 171/331] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:16.995517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:17.045097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 161/332] 10.222.0.99 () {58 vars in 1334 bytes} [Fri Sep 2 21:02:16 2022] PUT /identity/v3/system/users/14c102ffacb54cfa8393c88fda6b7961/roles/c219c1a78e09431ab0ac1b3968aff0b5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:17.129930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:17.190576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fc0e-313c-4fe2-a16f-31e8496dd624 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:17.262395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:17.264078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:17.264960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:17.265814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] PATH_INFO: `/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:17.268476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] RBAC: Authorizing `identity:get_user(user_id=b5fa74ca55f64396aa6c7f08c53be9e7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:17.271674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a45e-af18-4012-867c-1dfb04937374 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:17.275366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 172/333] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:16 2022] GET /identity/v3/users/b5fa74ca55f64396aa6c7f08c53be9e7 => generated 293 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:17.282352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:17.453879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:17.473380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf687ee4-f069-4072-8e1c-9cdba841715a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:17.518002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:17.518616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:17.518983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:17.519258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:17.520000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] RBAC: Adding query filter params (role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, user.id=b5fa74ca55f64396aa6c7f08c53be9e7, scope.system=all) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:17.520304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:17.521786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbf974-38f1-4b3f-81fa-f6e7a0396cbd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:17.539002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 162/334] 10.222.0.99 () {56 vars in 1308 bytes} [Fri Sep 2 21:02:17 2022] GET /identity/v3/role_assignments?user.id=b5fa74ca55f64396aa6c7f08c53be9e7&scope.system=all&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 510 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:18.524161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b12ee-1161-45c3-95e2-e3cdfd9cc22e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:18.525852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b12ee-1161-45c3-95e2-e3cdfd9cc22e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:18.526668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b12ee-1161-45c3-95e2-e3cdfd9cc22e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:18.529445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 173/335] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:18.535475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982086-2808-4f1d-9684-1fcc452ab759 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:18.536221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982086-2808-4f1d-9684-1fcc452ab759 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:18.536221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b982086-2808-4f1d-9684-1fcc452ab759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:18.571721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9b982086-2808-4f1d-9684-1fcc452ab759 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:18.574916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9b982086-2808-4f1d-9684-1fcc452ab759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:18.714234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 163/336] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:19.225220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43489ed-2a74-4460-b778-536691c7f7e8 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:19.225630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43489ed-2a74-4460-b778-536691c7f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:19.225857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43489ed-2a74-4460-b778-536691c7f7e8 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:19.227209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 174/337] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:19.235547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:19.235840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:19.236110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:19.268689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:19.270659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:19.339836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43489ed-2a74-4460-b778-536691c7f7e8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:19.415267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 164/338] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:19.533403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43489ed-2a74-4460-b778-536691c7f7e8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:19.582693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:19.583735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:19.584235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:19.584718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:19.593634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:19.602464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:19.608957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:19.609904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:19.611201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:19.614098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 175/339] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:19.620677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:19.623830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:19.828279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1242f-46a5-4318-9e1d-fa5a09a4e04e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:19.907286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:19.908312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:19.908849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:19.909327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:19.910573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:19.911112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:19.913689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:19.937309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 165/340] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:19.944441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:20.121313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:20.157604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6a192-2346-45e8-895c-5720e1ed8e83 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:20.206570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:20.210270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.210876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.211372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.246594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:20.330339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:20.333475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:20.335324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179ae21-2f54-47b0-b679-7f3c8560933f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:20.360204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5bcf8-fe86-42df-a91f-2e6a4e1182d7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:20.450634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:20.451207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.451490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.451762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.482509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b3c51025-53e8-461b-8de0-f13c902a76ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:20.488833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:20.490356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:20.491166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:20.495010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 166/341] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:20.499616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9342-caaa-4f0e-84c4-77abbc9421b3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.499983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9342-caaa-4f0e-84c4-77abbc9421b3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.500156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b9342-caaa-4f0e-84c4-77abbc9421b3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.507605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 167/342] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:20.509165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b9342-caaa-4f0e-84c4-77abbc9421b3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:20.529031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 176/343] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:20.530810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.531330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.531832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.556573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:20.559098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:20.673776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 177/344] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:20.674730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:20.775164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:20.862413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b9342-caaa-4f0e-84c4-77abbc9421b3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:20.913202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:20.931406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:20.932086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.932348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.932621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.933285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:20.933577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:20.934999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69077cf8-e1bb-4227-a3d1-af01c933b3fc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:20.951439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 168/345] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:20.957449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a03ae6-3051-414b-b160-fcbe9391bf75 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.957894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a03ae6-3051-414b-b160-fcbe9391bf75 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.958046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a03ae6-3051-414b-b160-fcbe9391bf75 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.959381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 169/346] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:20.960546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7b18-cc0b-484d-b12f-109d7666f678 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:20.960820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7b18-cc0b-484d-b12f-109d7666f678 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:20.961084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d7b18-cc0b-484d-b12f-109d7666f678 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:20.964281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ae040-68ea-4401-a3d3-d7925aa24c6a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:20.969571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 170/347] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:20.970558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7b18-cc0b-484d-b12f-109d7666f678 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:21.026720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:21.028050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.028558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.029038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.031620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:21.034544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709bbcd-83c5-40ae-8047-82ffb7f4c315 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:21.036995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 178/348] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:20 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:21.038607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f712d-de3c-42a1-a26d-6136e0d5700b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.039160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f712d-de3c-42a1-a26d-6136e0d5700b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.039658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f712d-de3c-42a1-a26d-6136e0d5700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.082557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-503f712d-de3c-42a1-a26d-6136e0d5700b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:21.085003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-503f712d-de3c-42a1-a26d-6136e0d5700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:21.192587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 179/349] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:21.196664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.197207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.199200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.200806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d7b18-cc0b-484d-b12f-109d7666f678 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:21.240129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:21.240129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:21.279503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:21.284116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:21.286011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.286011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.286193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.382366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 180/350] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:21.383678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:21.418302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:21.442611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:21.445382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:21.627805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6bd5e-7da8-459b-9c4b-88d0de1564d5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:21.675225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 171/351] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:21.676301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:21.691834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:21.692862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.693885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.704479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:21.706467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:21.708649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 181/352] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:21 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:21.709901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:21.736121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:21.799498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:21.823577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1fcce-4969-43b3-9abc-dcdce9dfaa90 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:21.830044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e990f2-3e70-4fd8-bef4-b8cabc9be031 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:21.884274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:21.886641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.886641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.886812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.890487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:21.891582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:21.892069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:21.893549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:21.910403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:21.918399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:21.919479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 21:02:21.920604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:21.921514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:21.921641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 182/353] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:21.921641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:21.928540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:21.930030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:21.931109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:21.935499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 172/354] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:21.937221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.048207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:22.074778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca827108-4d00-419e-a142-b0ff58de044b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.085979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f254-c2cf-4fb9-9431-4a80fc2c8474 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.126357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.127028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.127308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.127580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.128863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.130416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.132342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 183/355] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:21 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.133036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.153605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.154592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.154592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.154876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.155601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=90460bb6976b4887b7303a9cbeec4b38) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:22.155915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.157411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.166164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 173/356] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:21 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.167681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.193302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:22.312741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2dc55-13d6-4a72-b5d1-92f460bbdffe None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.362350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08432c-3c1f-4681-849e-05e8cffb7d4f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.368703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.370235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.370520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.370795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.386149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:22.391917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:22.392870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:22.394149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 184/357] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:22.395119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.409624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:22.435059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.436188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.436677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.437153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.438311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:22.438833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.441224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f475e1-629e-4ef2-a687-c13ffeb782ca None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.464147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 174/358] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.467267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce556-8953-414b-a5af-0c7a4f851051 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.467754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce556-8953-414b-a5af-0c7a4f851051 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.468217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce556-8953-414b-a5af-0c7a4f851051 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.473930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 175/359] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:22.474853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce556-8953-414b-a5af-0c7a4f851051 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.549973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3a862-f949-477b-936b-1bc3e16327e4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.618213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:22.635442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.636451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.636451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.636451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.636451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:22.636451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.639581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.652373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 185/360] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.653314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.685408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce556-8953-414b-a5af-0c7a4f851051 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.760642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.761661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.762538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.762805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.764382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.766169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.767924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 176/361] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.769537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:22.772847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:22.820479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac27ec-88a3-41fc-94aa-22f334e4eb66 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.871225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.871963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.872245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.872819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.873499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:22.874102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.875966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510534e-e3cb-4682-ade7-73bdccce4c8f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.889131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 186/362] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.890254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1385f64-e553-466c-a071-901b31593306 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.890514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1385f64-e553-466c-a071-901b31593306 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.890788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1385f64-e553-466c-a071-901b31593306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.917509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d1385f64-e553-466c-a071-901b31593306 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:22.918764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d1385f64-e553-466c-a071-901b31593306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:22.933695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072decec-7b8a-4829-a0d7-0ff63ba65d3c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:22.981545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:22.982244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:22.982585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:22.982882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] PATH_INFO: `/v3/users/14c102ffacb54cfa8393c88fda6b7961` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:22.984074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] RBAC: Authorizing `identity:get_user(user_id=14c102ffacb54cfa8393c88fda6b7961)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:22.985645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:22.993771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 177/363] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/users/14c102ffacb54cfa8393c88fda6b7961 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:22.993771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:23.005848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:23.041656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 187/364] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:23.042916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1385f64-e553-466c-a071-901b31593306 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:23.203969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:23.224658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324a68b-a6e7-4580-9336-3747a38b06ec None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:23.312090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:23.312660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:23.312935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:23.313203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:23.324557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1385f64-e553-466c-a071-901b31593306 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:23.405064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:23.406807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:23.407152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:23.407484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:23.408306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=90460bb6976b4887b7303a9cbeec4b38) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:23.412380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:23.412380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:23.419175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 188/365] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:23 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 260 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:23.420728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:23.438632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:23.441163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:23.456370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:23.621820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 178/366] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:23.623389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:23.631189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c11b63-829e-4a10-803e-816e3f84df95 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:23.686048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:23.708636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:23.709818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:23.710331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:23.710799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:23.824718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602d681-a550-4c27-9cfd-172c0d525237 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:23.846270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:23.858989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:23.861032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:23.920278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:23.921870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:23.922154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:23.922426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:23.923225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] RBAC: Adding query filter params (role.id=c219c1a78e09431ab0ac1b3968aff0b5, user.id=14c102ffacb54cfa8393c88fda6b7961, scope.system=all) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:23.924388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:23.925087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:23.941525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 179/367] 10.222.0.99 () {56 vars in 1308 bytes} [Fri Sep 2 21:02:23 2022] GET /identity/v3/role_assignments?user.id=14c102ffacb54cfa8393c88fda6b7961&scope.system=all&role.id=c219c1a78e09431ab0ac1b3968aff0b5 => generated 510 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:23.942475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.095420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 189/368] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:24.096404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.161907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6f9d-65bc-4e86-9cc6-f4507feb719a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.190239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:24.260249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.260895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.261171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.261440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.274856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:24.283445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:24.284426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:24.285233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 180/369] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:24.285944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.351053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4183c-3374-4077-b204-cc76990244ff None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.365157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:24.400841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.401907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.402328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.402741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.404585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:24.406309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:24.408455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 190/370] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:24.409419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.544247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31ac3-ceb1-4aaf-8a22-3ee116d6c276 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.548531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:24.558891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b142304-0cb2-4a38-bfc1-9fa99a037ce8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.607721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.608685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.609159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.609652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.610556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:24.611055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:24.612862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:24.623804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 191/371] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:24.629475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.629682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.629819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.629819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.629819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.629819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:24.632396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:24.638901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 181/372] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:24.640370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.822578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:24.831551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096b7bf-4771-41c3-a968-365915963e32 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.838751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de012884-bf99-4d34-b7f3-5ae7440cef9a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:24.901297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.902395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.902879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.903352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.905376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:24.908251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:24.911962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 192/373] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:24.913381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:24.928539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:24.929659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:24.930189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:24.930674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:24.976075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:25.076471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:25.079258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618402a-bd0a-4d7e-81b5-5cd202500da9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:25.095118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d2906-f05e-4412-8157-f30cfa31b30e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:25.172790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:25.174442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.174442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.174442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.175545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:25.177147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:25.178954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 193/374] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:25.179834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:25.255228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:25.268719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 182/375] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:25.270890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e134a2-e304-4f9d-a280-053f9dd6bc7e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.271388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e134a2-e304-4f9d-a280-053f9dd6bc7e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.271908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e134a2-e304-4f9d-a280-053f9dd6bc7e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.280789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 183/376] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:25.282442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e134a2-e304-4f9d-a280-053f9dd6bc7e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:25.304443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:25.314098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b967fb0-c9e4-4d2a-93e3-29cb58642c9b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:25.354509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:25.355036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.355312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.355638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.356388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=f54be7acf91e47c5bd233f1914534020) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:25.356687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:25.358204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:25.364601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 194/377] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:25 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:25.365553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:25.489326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e134a2-e304-4f9d-a280-053f9dd6bc7e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:25.558997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a624479-a832-42b4-b745-ffdd34e2e6fa None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:25.569649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:25.571528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.572399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.573057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.574721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=f54be7acf91e47c5bd233f1914534020) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:25.577075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:25.580102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f0d0f-9bab-48b8-b089-71f6c0e4ab6c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:25.593018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 184/378] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:25 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:25.596184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.596965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.597994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.644876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:25.647023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:25.656359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:25.657363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.657872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.658377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] PATH_INFO: `/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:25.660456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] RBAC: Authorizing `identity:get_project(project_id=8077ff7239e7425ab3b0ae8d4fecdd2f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:25.663996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:25.667679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 195/379] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:25 2022] GET /identity/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:25.679709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:25.749789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:25.772822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:25.776371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 185/380] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:25.924156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e2927-0e34-4e88-bd62-f1f624dc24bc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:25.992901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:25.994048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:25.994577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:25.995059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] PATH_INFO: `/v3/groups/9c8b37a4cef54adf82f30e5d7838778d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:26.004414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] RBAC: Authorizing `identity:get_group(group_id=9c8b37a4cef54adf82f30e5d7838778d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:26.007127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cccb772-4a52-49af-bffe-e8dd01b0547b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:26.010781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 196/381] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:25 2022] GET /identity/v3/groups/9c8b37a4cef54adf82f30e5d7838778d => generated 229 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:26.021624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:26.085962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb1488-f706-4cc5-b108-c7c13743944b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:26.086881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb1488-f706-4cc5-b108-c7c13743944b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:26.088498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb1488-f706-4cc5-b108-c7c13743944b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:26.096431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 197/382] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:26.105037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:26.105997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:26.106743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:26.167013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:26.168244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:26.214418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d397776-39da-40b8-bc59-e865f79485fe None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:26.290371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:26.291636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:26.291943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:26.292338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:26.293626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] RBAC: Adding query filter params (group.id=9c8b37a4cef54adf82f30e5d7838778d, role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=8077ff7239e7425ab3b0ae8d4fecdd2f) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:26.294177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:26.296839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:26.307575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 186/383] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:26 2022] GET /identity/v3/role_assignments?group.id=9c8b37a4cef54adf82f30e5d7838778d&scope.project.id=8077ff7239e7425ab3b0ae8d4fecdd2f&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 260 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:26.350288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:26.391611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 198/384] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:26.497994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:26.625739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214584-d8a6-4124-a238-6864e9206571 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:26.703615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:26.704657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:26.705128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:26.705591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:26.726037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:26.736589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:26.738382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:26.740148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 187/385] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:26.751490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:26.815454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:26.961140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:27.023429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da966b7-8343-4187-9a82-18b0d6ef840c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:27.115659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:27.116407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:27.116698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:27.116949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:27.118112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:27.118112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:27.120082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:27.162242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 199/386] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:27.169281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:27.199951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee0c07-93b0-4167-ad77-dace7d28e818 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:27.247783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:27.301780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:27.302751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:27.303241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:27.303725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:27.356087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:27.377842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:27.378799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:27.379572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:27.380734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 188/387] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:27.395759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:27.408733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:27.481327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefac2e-4bee-4725-bf55-dd64d48d98fd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:27.565797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:27.569741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:27.569741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:27.569741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:27.569971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:27.573285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:27.583380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 200/388] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:27 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:27.593696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:27.832861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9674-ef93-4975-af54-f1317d168da7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:27.964525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:27.965744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:27.969746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:27.969746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:27.969746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:27.969746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:27.970343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:28.004167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e798d9-964a-43b0-ad67-25ea237d02a0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:28.014633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 189/389] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 620 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:28.028693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:28.141840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:28.141840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.141840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.141840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.155875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:28.160476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:28.179890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 201/390] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:27 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:28.185756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:28.207093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:28.264592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:28.341692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99269c25-2bbb-46ed-aeef-e0460ecf4de4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:28.409465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:28.410785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.411409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.412038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.414200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:28.418266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669061e-a5c1-4f42-a0ad-6857556dc48d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:28.422283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 190/391] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:28 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:28.424741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d2c27-ef9e-4b71-b3e4-32bd12f40865 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.425386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d2c27-ef9e-4b71-b3e4-32bd12f40865 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.426050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d2c27-ef9e-4b71-b3e4-32bd12f40865 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.429167 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 191/392] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:28.435038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d2c27-ef9e-4b71-b3e4-32bd12f40865 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:28.466966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbd147-bc01-4d8e-bc3c-b9507eb14a54 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:28.513839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:28.514771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.515193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.515636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.518416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, group_id=55c09ddc0fd540d0af7b957c129b2e6c, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:28.520845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb7772-6508-45b6-a55b-14e0dd80dd3a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:28.596967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d2c27-ef9e-4b71-b3e4-32bd12f40865 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:28.598957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:28.623719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 202/393] 10.222.0.99 () {58 vars in 1442 bytes} [Fri Sep 2 21:02:28 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:02:28.626324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501ce5-e0fd-4d5d-9f5d-1c8884c0b645 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.627289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501ce5-e0fd-4d5d-9f5d-1c8884c0b645 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.627810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04501ce5-e0fd-4d5d-9f5d-1c8884c0b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.683539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:28.684555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.685020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.685506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.689220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-04501ce5-e0fd-4d5d-9f5d-1c8884c0b645 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:28.692212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-04501ce5-e0fd-4d5d-9f5d-1c8884c0b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:28.693894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:28.700739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:28.704379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 192/394] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:28 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:28.720701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:28.861822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 203/395] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:28.953375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f146a18-33da-4f80-8dfe-d5256ac9b890 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:28.961519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2378b-ddc4-4460-b925-7be9bdd17d8e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.962044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2378b-ddc4-4460-b925-7be9bdd17d8e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.963316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2378b-ddc4-4460-b925-7be9bdd17d8e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:28.964927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 204/396] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:28.970639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47268a78-3761-4975-ace6-13e3c449a0ff None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:28.970914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47268a78-3761-4975-ace6-13e3c449a0ff None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:28.971189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47268a78-3761-4975-ace6-13e3c449a0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.001967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:29.011706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-47268a78-3761-4975-ace6-13e3c449a0ff None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:29.023574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-47268a78-3761-4975-ace6-13e3c449a0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:29.023772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477793-ecad-422e-9480-20b621eea450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:29.023772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477793-ecad-422e-9480-20b621eea450 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.023772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477793-ecad-422e-9480-20b621eea450 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.023772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477793-ecad-422e-9480-20b621eea450 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.024259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477793-ecad-422e-9480-20b621eea450 None admin] RBAC: Authorizing `identity:create_grant(project_id=90460bb6976b4887b7303a9cbeec4b38, group_id=55c09ddc0fd540d0af7b957c129b2e6c, role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:29.026381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477793-ecad-422e-9480-20b621eea450 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:29.101981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 205/397] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:29.103219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478973-0852-4ad5-9513-5c76f4ddd6f2 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.103496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478973-0852-4ad5-9513-5c76f4ddd6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.103755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6478973-0852-4ad5-9513-5c76f4ddd6f2 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.105325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 206/398] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:29.113785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.113785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.113785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.129013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 193/399] 10.222.0.99 () {58 vars in 1442 bytes} [Fri Sep 2 21:02:28 2022] PUT /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/44671b36ffe9406e9372616fdd4b9400 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:29.138804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:29.140022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:29.171716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477793-ecad-422e-9480-20b621eea450 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:29.271682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:29.325143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 207/400] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:29.409351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:29.429526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477793-ecad-422e-9480-20b621eea450 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:29.510843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:29.512589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.513402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.514754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.554471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:29.577025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:29.578813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:29.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:29.587111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 194/401] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:29.595427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:29.644445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:29.710950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd425d9-7f8c-4774-bf43-1b27f75d8ba9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:29.774237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:29.775259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.775617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.775918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.793566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:29.802060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:29.803007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:29.803791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:29.804385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 208/402] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:29.809835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:29.835192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:29.854979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a12ab0-3fc2-4799-8ea5-b987b5fe65cb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:29.925536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:29.926758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:29.927409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:29.928038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:29.929182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:29.929962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:29.934389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:29.952566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 195/403] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:29.954623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.018450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dbbb3-4256-459e-a17d-a62d78fdcafe None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.074419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:30.090356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.092143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.092143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.092435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.114172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:30.125224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:30.127212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:30.129705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:30.133627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 209/404] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:30.136125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.169844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caadaf-ad08-40ec-9552-59263816dcd5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.239888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.246915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.247859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.248470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.250003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:30.250771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.253629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.271597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 196/405] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.273404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.306614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:30.351578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819c05-8d94-47ba-8ee1-b966acb3dee7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.404866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.405493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.405808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.406082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.407792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.410297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.412551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 210/406] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.414391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.481166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:30.484618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab4569-deac-467d-8620-30e92863ff96 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.531890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a212fb6-e1dd-4d84-bb23-aeb7ea961a2d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.558276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.559653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.560372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.561078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.562663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:30.563426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.566378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.570387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.571028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.571301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.572233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.573566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.575150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.581393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 211/407] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.582345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.586363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 197/408] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.588846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.700953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23329-cf8f-4992-adb7-673ceb9eb218 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.728081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:30.741423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.742344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.742459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:30.742892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.743363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.745350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.747382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.749599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 212/409] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.750625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.791480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623a991-d1ae-4922-957f-a4e83e2d4512 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.859730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.860796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.861268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.861749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.863868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.866678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.876646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 198/410] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.876646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:30.910462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783359b5-eb75-4d01-839f-4c3af3324944 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:30.957791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:30.959135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:30.959694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:30.960207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] PATH_INFO: `/v3/groups/9c8b37a4cef54adf82f30e5d7838778d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:30.961876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] RBAC: Authorizing `identity:get_group(group_id=9c8b37a4cef54adf82f30e5d7838778d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:30.964705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:30.967423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 213/411] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:30 2022] GET /identity/v3/groups/9c8b37a4cef54adf82f30e5d7838778d => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:30.968860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:31.060939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:31.081273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485253fd-31be-45b8-8a1d-66c6f6773677 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:31.116245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38f291-126c-4f1d-a88d-583cbea8158d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:31.150939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:31.151977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.152465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.152934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.156982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, group_id=55c09ddc0fd540d0af7b957c129b2e6c, role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:31.159925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:31.167886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:31.169162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.169788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.170399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.173176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f54be7acf91e47c5bd233f1914534020, group_id=55c09ddc0fd540d0af7b957c129b2e6c, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:31.174801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca62b1b-67bc-4e77-89cb-1b6c888a75f6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:31.297097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 199/412] 10.222.0.99 () {58 vars in 1442 bytes} [Fri Sep 2 21:02:30 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/44671b36ffe9406e9372616fdd4b9400 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:31.298089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:31.301973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:31.338323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 214/413] 10.222.0.99 () {58 vars in 1442 bytes} [Fri Sep 2 21:02:30 2022] PUT /identity/v3/projects/f54be7acf91e47c5bd233f1914534020/groups/55c09ddc0fd540d0af7b957c129b2e6c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:31.340877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518893a8-9c76-4a95-af45-e1ed485e7eaa None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.341465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518893a8-9c76-4a95-af45-e1ed485e7eaa None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.342201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518893a8-9c76-4a95-af45-e1ed485e7eaa None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.344616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 215/414] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:31.354378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe19b29-6481-4abf-95b7-16df2d85a583 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.355239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe19b29-6481-4abf-95b7-16df2d85a583 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.355239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe19b29-6481-4abf-95b7-16df2d85a583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.399314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9fe19b29-6481-4abf-95b7-16df2d85a583 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:31.401959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9fe19b29-6481-4abf-95b7-16df2d85a583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:31.479653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd26821-cdf2-4ef9-bc76-b8687d0fac90 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:31.546417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:31.548673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.549285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.549984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] PATH_INFO: `/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.550560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 216/415] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:31.552242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] RBAC: Authorizing `identity:get_project(project_id=8077ff7239e7425ab3b0ae8d4fecdd2f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:31.553894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978c22-aa0b-4846-a8cd-3e2f57d1be17 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.554528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978c22-aa0b-4846-a8cd-3e2f57d1be17 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.554671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:31.555154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3978c22-aa0b-4846-a8cd-3e2f57d1be17 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.557971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 200/416] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:31 2022] GET /identity/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:31.561833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 217/417] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:31.566344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:31.568143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.568573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.569073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.602505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:31.604636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:31.706289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:31.745056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:31.764597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36589-7915-41b5-a79a-c06dcc1a7153 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:31.830004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:31.831084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:31.831546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:31.832012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] PATH_INFO: `/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f/groups/9c8b37a4cef54adf82f30e5d7838778d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:31.835346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] RBAC: Authorizing `identity:create_grant(project_id=8077ff7239e7425ab3b0ae8d4fecdd2f, group_id=9c8b37a4cef54adf82f30e5d7838778d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:31.838604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:31.847736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 218/418] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:31.926331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:31.938575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 201/419] 10.222.0.99 () {58 vars in 1442 bytes} [Fri Sep 2 21:02:31 2022] PUT /identity/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f/groups/9c8b37a4cef54adf82f30e5d7838778d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:02:32.091915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e747ae-43cb-4265-ac40-2b2713908db8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:32.159572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:32.160712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:32.161188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:32.161644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:32.180401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:32.188595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:32.189563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:32.190368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 219/420] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:32.198937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:32.257211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:32.334885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232844e-ad37-4856-acdc-033e78a6d243 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:32.352177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:32.430120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:32.454265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:32.455337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:32.455825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:32.456302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:32.457459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:32.458008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:32.460542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:32.488506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 202/421] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:32.495051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:32.512080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34cf9-bed0-4c08-808c-8d3a9f6daeaf None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:32.642509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:32.643175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:32.643456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:32.643722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:32.676845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:32.685663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:32.687317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:32.688957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:32.699138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a2de-e285-4c4d-a91c-419adbb09d3a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:32.703163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 220/422] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:32.709036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:32.776737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:32.777656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:32.778120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:32.778407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:32.943077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:32.947059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:32.954089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:32.978106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:32.996086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eafe3-41d3-4b42-a409-79145ea6982c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:33.143116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 203/423] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:33.151555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:33.154425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:33.155062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:33.155346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:33.155626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:33.156327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:33.156632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:33.158061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:33.178958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 221/424] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:33.182502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:33.324762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:33.332975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344f5e9-7397-47ea-b660-a34d32a7c0a4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:33.377162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:33.378454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:33.378874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:33.379158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:33.380556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:33.382239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:33.384383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 204/425] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:33.393125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:33.424278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bcb5b-2457-46af-a8e9-4fc1602c3a24 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:33.486800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:33.487416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:33.487696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:33.487970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:33.522766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:33.576754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:33.579140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:33.722055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 222/426] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:02:33.723707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f3de6-e42f-400f-824a-98467863a4cb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:33.732882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:33.824637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:33.832128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:33.832992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:33.833401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:33.833853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:33.838482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:33.840173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:33.842245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 205/427] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:33.851645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:33.874078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add07d50-b4a6-41a5-a356-f38f884686da None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:33.917956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:33.918739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:33.919312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:33.919723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] PATH_INFO: `/v3/projects/90460bb6976b4887b7303a9cbeec4b38` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:33.921351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=90460bb6976b4887b7303a9cbeec4b38)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:33.923459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:33.925980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 223/428] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/projects/90460bb6976b4887b7303a9cbeec4b38 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:33.927795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:33.935384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:34.064150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e24de-faa5-4ab7-b5cf-f7e0af53ac85 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:34.130524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:34.131302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.131679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.131957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.133124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=90460bb6976b4887b7303a9cbeec4b38) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:34.133477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:34.135805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c078-8b5c-42ae-bf2a-800d9c98b82b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:34.137246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903ea9d-b02d-470c-a410-7b76486ab0e5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:34.145449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 206/429] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 611 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:34.147615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90817-9a78-4e7e-b38b-f6ca5a285fd7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.147772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90817-9a78-4e7e-b38b-f6ca5a285fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.148123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e90817-9a78-4e7e-b38b-f6ca5a285fd7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.150232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 207/430] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:34.157433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5c7e-86b4-4088-a164-d113c26651a3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.157433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5c7e-86b4-4088-a164-d113c26651a3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.157977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c5c7e-86b4-4088-a164-d113c26651a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.175292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:34.187552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:34.188477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-382c5c7e-86b4-4088-a164-d113c26651a3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:34.188790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.190187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.190187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.190378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-382c5c7e-86b4-4088-a164-d113c26651a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:34.191448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:34.193629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:34.197491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 224/431] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:33 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:34.204770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:34.300596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 208/432] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:34.391509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9e3ad-c582-4332-a161-7595c27c3901 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:34.422641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad29d7a-a137-4ea6-9ce1-fded4b428341 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.423759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad29d7a-a137-4ea6-9ce1-fded4b428341 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.424230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad29d7a-a137-4ea6-9ce1-fded4b428341 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.426190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 209/433] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:34.439868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.440541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.441454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.459286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273502-8497-4448-b198-e93119985d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:34.460339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273502-8497-4448-b198-e93119985d69 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.460819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273502-8497-4448-b198-e93119985d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.461294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273502-8497-4448-b198-e93119985d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.463354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273502-8497-4448-b198-e93119985d69 None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=90460bb6976b4887b7303a9cbeec4b38) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:34.463897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273502-8497-4448-b198-e93119985d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:34.466725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273502-8497-4448-b198-e93119985d69 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:34.473227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:34.475182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:34.480634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 225/434] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:34 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=90460bb6976b4887b7303a9cbeec4b38&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 611 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:34.501835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba52250-5531-4b19-b184-f53bd25daf78 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.502320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba52250-5531-4b19-b184-f53bd25daf78 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.503053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba52250-5531-4b19-b184-f53bd25daf78 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.505876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 226/435] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:34.513681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:34.514188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:34.514650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:34.528421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:34.554437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:34.557201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:34.596768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 210/436] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:34.658529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:34.675691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 227/437] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:34.770429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:34.949514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a34167-9be2-4804-b2bf-362478e57387 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:34.989521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.000763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.002110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.002392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.002886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.025314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:35.035647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:35.036581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:35.037423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:35.038072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 211/438] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:35.047517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.078175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.178697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc383b4-b4c8-43e1-ac79-deb90c1d2820 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.204738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ad855-54a2-4d07-a318-d9c75554d23a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.242419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.243935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.244420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.244829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.256137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.256767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.257041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.257308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.258048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:35.258347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:35.260518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:35.270302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:35.278359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 212/439] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:35.278905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:35.279805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 21:02:35.280620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:35.281241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.281312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 228/440] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:35.284565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.349971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.408403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c0f18-7b9e-4a96-aa7d-ced779148bcc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.409836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.445584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.446258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.447667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.448267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.450646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc0f6-4d11-44f0-8fa9-2471630dee96 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.500462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.501007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.501007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.501330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.513886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:35.519443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:35.521129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 213/441] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:35.523233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.549922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:35.552595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:35.636435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.655179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4ce59-151b-4c44-98cf-1dc87ee9f1de None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.712208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.713627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.714025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.714375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.715294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:35.715689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:35.717334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:35.718832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 229/442] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:02:35.720655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.724687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.730649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 214/443] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:35.732099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.860467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9361a18-4cc5-44af-8738-b2eaef769b31 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.865909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e6a7a-fe19-4274-bd13-251ee3c0d8fe None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:35.900921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.902032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.902531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.903006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.905164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:35.907120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:35.907773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:35.908763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:35.908923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 215/444] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:35.909057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:35.909457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:35.909745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.910479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:35.910895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:35.912384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:35.922194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 230/445] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:35.923038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:35.941249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:35.999644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.041180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878baa4-ed7b-4f06-8794-e1b1778be992 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.065571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15f20-0019-461f-8c6e-552273a8214e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.081286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.082065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.082433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.082805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.115455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.116057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.116379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.116730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.118027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.121008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.123202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 231/446] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:36.124261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.177248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.179804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.247749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.273130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1696-26e5-4d72-aee5-ece2ba9bf533 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.277290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.317633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.318482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.318846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.319202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.322464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 216/447] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:36.323222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.408936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.410359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.458612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16e9d4-1c5a-40df-8536-0541235332e5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.508522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.510783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.511233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.511594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.512423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=f54be7acf91e47c5bd233f1914534020) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:36.512795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.514574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.518102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.522900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 217/448] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:36.523746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.543810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 232/449] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:36.551288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.574576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.677162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb261e-c107-43a1-884d-60fa013ce923 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.713990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f386c20-dd5d-4d75-b1b3-64b05ac6fa40 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.735758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.737068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.737577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.738112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] PATH_INFO: `/v3/projects/f54be7acf91e47c5bd233f1914534020` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.740234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] RBAC: Authorizing `identity:get_project(project_id=f54be7acf91e47c5bd233f1914534020)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.743036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.746946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 218/450] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/projects/f54be7acf91e47c5bd233f1914534020 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:36.758666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.765351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.766112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.768170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.768170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] PATH_INFO: `/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.768170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] RBAC: Authorizing `identity:get_project(project_id=8077ff7239e7425ab3b0ae8d4fecdd2f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.770176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.771247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 233/451] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/projects/8077ff7239e7425ab3b0ae8d4fecdd2f => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:36.781184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.873898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.916460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:36.920921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0472cf4-1535-4d8e-9587-1185ed4b15c7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.933152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6278a-932f-4507-af51-359307bbbae0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:36.959133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.959779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.960136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.960340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] PATH_INFO: `/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.962475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] RBAC: Authorizing `identity:get_group(group_id=55c09ddc0fd540d0af7b957c129b2e6c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.964314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.965995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 219/452] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/groups/55c09ddc0fd540d0af7b957c129b2e6c => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:36.972360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:36.972993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:36.973091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:36.973273 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:36.973548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] PATH_INFO: `/v3/groups/9c8b37a4cef54adf82f30e5d7838778d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:36.974718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] RBAC: Authorizing `identity:get_group(group_id=9c8b37a4cef54adf82f30e5d7838778d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:36.976169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:36.978376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 234/453] 10.222.0.99 () {56 vars in 1180 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/groups/9c8b37a4cef54adf82f30e5d7838778d => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:36.984552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:37.107096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff0f84-ffd6-448d-9a5b-068dc757299e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:37.120800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f43b17-0dc6-4a3c-9257-e9a4f133ce18 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:37.126442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:37.145757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:37.146368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:37.146650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:37.146916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:37.147651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] RBAC: Adding query filter params (group.id=9c8b37a4cef54adf82f30e5d7838778d, role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=8077ff7239e7425ab3b0ae8d4fecdd2f) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:37.147955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:37.150133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8d2a7-7ca0-4680-ae3b-972a42762e2b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:37.160070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 235/454] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/role_assignments?group.id=9c8b37a4cef54adf82f30e5d7838778d&scope.project.id=8077ff7239e7425ab3b0ae8d4fecdd2f&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:37.172344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:37.172974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:37.173269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:37.173546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:37.174353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] RBAC: Adding query filter params (group.id=55c09ddc0fd540d0af7b957c129b2e6c, role.id=e1924b0c1a3541d0a78b3f6ce099d6dc, scope.project.id=f54be7acf91e47c5bd233f1914534020) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:37.174721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:37.176211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8a86-fa1e-4cc5-8db3-f6b5912bf87a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:37.183482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 220/455] 10.222.0.99 () {56 vars in 1376 bytes} [Fri Sep 2 21:02:36 2022] GET /identity/v3/role_assignments?group.id=55c09ddc0fd540d0af7b957c129b2e6c&scope.project.id=f54be7acf91e47c5bd233f1914534020&role.id=e1924b0c1a3541d0a78b3f6ce099d6dc => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:40.306621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460da0-7190-4b70-8606-b4c1425be5bf None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:40.307181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460da0-7190-4b70-8606-b4c1425be5bf None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:40.307259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b460da0-7190-4b70-8606-b4c1425be5bf None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:40.308903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 236/456] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:40.320314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61147f6-7c9f-4f55-993e-6b8876cdf57e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:40.320879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61147f6-7c9f-4f55-993e-6b8876cdf57e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:40.321880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61147f6-7c9f-4f55-993e-6b8876cdf57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:40.357253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e61147f6-7c9f-4f55-993e-6b8876cdf57e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:40.359236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e61147f6-7c9f-4f55-993e-6b8876cdf57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:40.415400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:40.491613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 221/457] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:40.868516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9782720-186e-45a0-a85a-58f8eb0dc661 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:40.868516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9782720-186e-45a0-a85a-58f8eb0dc661 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:40.868516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9782720-186e-45a0-a85a-58f8eb0dc661 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:40.870891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 237/458] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:40.877231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730529b-2462-4b65-a2b0-0f44e40e32c5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:40.877683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730529b-2462-4b65-a2b0-0f44e40e32c5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:40.878215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730529b-2462-4b65-a2b0-0f44e40e32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:40.912937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e730529b-2462-4b65-a2b0-0f44e40e32c5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:40.914203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e730529b-2462-4b65-a2b0-0f44e40e32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:40.938871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9782720-186e-45a0-a85a-58f8eb0dc661 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:41.077982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 222/459] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:41.178633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9782720-186e-45a0-a85a-58f8eb0dc661 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:41.233096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:41.234045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.234830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.234830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.251668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758673b-4145-44ce-b0f1-23fc745041e3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.252332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758673b-4145-44ce-b0f1-23fc745041e3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.252439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:41.252905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758673b-4145-44ce-b0f1-23fc745041e3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.253959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:41.257828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 223/460] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:41.261460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956115-adb5-4b81-8db2-c8e0b19f0f56 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.261764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956115-adb5-4b81-8db2-c8e0b19f0f56 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.262049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31956115-adb5-4b81-8db2-c8e0b19f0f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:41.274885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:41.276997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:41.278144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:41.278144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:41.278144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:41.278144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 238/461] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:41.282498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:41.294473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-31956115-adb5-4b81-8db2-c8e0b19f0f56 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:41.295692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-31956115-adb5-4b81-8db2-c8e0b19f0f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:41.354355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:41.380621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 224/462] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:41.456065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ad067-3414-49f1-a1e1-845390f22bcb None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:41.489892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964db756-7aaf-4835-9de1-d8b5776db368 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.490238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964db756-7aaf-4835-9de1-d8b5776db368 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.490439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964db756-7aaf-4835-9de1-d8b5776db368 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.491777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 225/463] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:41.500549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.500941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.501340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.515108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:41.516052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.516453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.516861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.517867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:41.518297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:41.520432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:41.551489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:41.553525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:41.555301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 239/464] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:41.564986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:41.583761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:41.702956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 226/465] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:41.703319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:41.706743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:41.713870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e821577-d238-43a8-8650-9b02d033542f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:41.754328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:41.755461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.756018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.756639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.757328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:41.758958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:41.842042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2b435-d226-4e4b-9737-e707d93beb95 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:41.882224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 240/466] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:41 2022] POST /identity/v3/users => generated 246 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:41.883223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:41.902747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:41.905409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.905751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.906173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:41.941624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:41.944584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8c4ffc6a-a59c-4307-ac6c-be6199b87aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:41.987808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:41.989669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:41.991728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:41.991728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:41.991728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:41.994831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 227/467] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:41.997490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837158c6-8f43-4568-89ff-3f710f1815c3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:41.998221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837158c6-8f43-4568-89ff-3f710f1815c3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:41.998899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837158c6-8f43-4568-89ff-3f710f1815c3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.005766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 228/468] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:42.005766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837158c6-8f43-4568-89ff-3f710f1815c3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.008760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:42.099960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6afc4-99e1-475b-8674-e3ba09c7bc31 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.161377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.161377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.161689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.161835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.167221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837158c6-8f43-4568-89ff-3f710f1815c3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.174653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.176148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.186250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3a6c24a6-ebac-4320-849c-7408976c7208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:42.194403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:42.204186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 241/469] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.205800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.217350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.218013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.218286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.218578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.219187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:42.219484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.220963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.221190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:42.223234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:42.243945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 229/470] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:42.243945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.329438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:42.330107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 242/471] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:42.331660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.398191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6735b-fd48-42f0-84da-87be93eacc35 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.459616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.460691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.461069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.461456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.474283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.476024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:42.494538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:42.496023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:42.497847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:42.497847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:42.497847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:42.498856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 230/472] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:42.499869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.567154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952f0ef-5325-485a-a603-aa05a5f2a5bf None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.631370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:42.642425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.653951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.677452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 243/473] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:42.679970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.694777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bf06f-0b7c-43ce-8a25-a0c2fc387c42 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.717197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:42.751061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.751578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.751956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.752306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.752866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.754359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.832271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2af53-78f6-481f-a7ce-4c88536f0b90 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:42.872563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 231/474] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:42 2022] POST /identity/v3/users => generated 248 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:42.873606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:42.910909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:42.913031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:42.914679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:42.916062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:42.917405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:42.918286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:42.920879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:42.948742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 244/475] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:42.950549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:43.041491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:43.100094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa27c60-bf42-470f-9770-8455703f4a05 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:43.131832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790f5f6-559b-4d5d-b5f5-e04de2e25c75 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:43.168706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:43.169580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:43.170037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:43.170449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:43.171466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:43.173461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:43.191317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:43.198333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:43.198333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:43.198333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:43.198333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:43.198333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:43.218099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.sql.core [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 21:02:43.218099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 21:02:43.218099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [parameters: {'id': '72c29f996b80418aa72390a589b9cf02', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 21:02:43.218099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81843) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application result.read() Sep 02 21:02:43.237350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return ref.to_dict() Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:02:43.239205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application yield resource Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise self.value Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application session.commit() Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 21:02:43.242359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self.session.flush() Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application compat.raise_( Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 21:02:43.249469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:02:43.252188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application result.read() Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:02:43.255268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [parameters: {'id': '72c29f996b80418aa72390a589b9cf02', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:43.258175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 21:02:43.262133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:43.268917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 232/476] 10.222.0.99 () {60 vars in 1131 bytes} [Fri Sep 2 21:02:42 2022] POST /identity/v3/roles => generated 144 bytes in 366 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:43.268917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:43.303251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 245/477] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:42 2022] POST /identity/v3/users => generated 248 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:43.303251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:43.332368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:43.469153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029803d-826e-4f27-8c37-1bed42dfc31b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:43.547716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:43.549127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:43.550138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:43.551041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:43.552833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:43.557440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:43.584569 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f79b4-77a2-46ca-8c88-7e745c0c2e67 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:43.599729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:43.657626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:43.658572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:43.658988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:43.659423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:43.676512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:43.677138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 233/478] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:43 2022] POST /identity/v3/users => generated 249 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-6864144e-f123-4d71-9fc3-dc02837ec6f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:43.686033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:43.687355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:43.688438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 246/479] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:02:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:43.694434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:43.819227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:43.830912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75dd0f-15ba-4c7a-ae87-e204d020a82f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:43.878835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:43.879935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:43.880407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:43.880866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:43.881781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:43.882268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:43.883944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5c32b-c99f-445d-9230-a313c5a39d1b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:43.896905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 234/480] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:44.491377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f73f7-088a-428b-a06d-8ef86df3e9e0 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:44.491781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f73f7-088a-428b-a06d-8ef86df3e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:44.492143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f73f7-088a-428b-a06d-8ef86df3e9e0 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:44.495511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 247/481] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:44.503825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:44.504425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:44.504425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:44.530902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:44.532132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:44.629759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 235/482] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:45.005860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f73f7-088a-428b-a06d-8ef86df3e9e0 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:45.125871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f73f7-088a-428b-a06d-8ef86df3e9e0 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:45.171779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:45.174659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.175455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.176939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.198413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-488255b4-ef7e-46d9-8a6d-3dcfbc1ceb6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:45.206352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:45.207352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:45.208614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:45.217871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 248/483] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:45.220945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:45.325427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:45.350443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cdcdb-b254-4efb-aa36-98737ff22ec7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:45.365864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbfda-17dc-4df8-bd28-c4b90a610bd5 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.366240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbfda-17dc-4df8-bd28-c4b90a610bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.366594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbfda-17dc-4df8-bd28-c4b90a610bd5 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.368799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 249/484] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:45.375678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.375981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.376230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.405481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:45.406998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:45.434084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:45.434724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.435001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.435271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.435953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:45.436258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:45.437656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:45.454123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:45.457878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 236/485] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:45.462275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:45.551391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 250/486] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:45.663169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f4a6-fa65-4b6d-b7b5-e35ed9c65f60 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:45.730480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:45.731333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.731612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.731881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.754159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:45.755618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:45.761885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:45.778560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:45.779539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:45.780379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:45.780379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:45.780379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:45.781061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 237/487] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:45.786065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:45.938184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20a396-ce78-4e51-b1ad-284c15051934 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:45.942564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:45.994694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:45.995328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:45.995603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:45.995867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:45.996718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:45.997413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:45.999136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.022900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 251/488] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:46.027970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.066084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:46.170117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5c680-f376-4cbc-85a6-30ff925f4acc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.211130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.211724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.211997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.212276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.219242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ca4a2-b183-4821-8f60-6213302e7419 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.223594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:46.229578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:46.230500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:46.231272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 238/489] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:46.239639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.281924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.282571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.282841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.283103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.296517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.298016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b44460b8-e22a-4066-b103-298b420d9172 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:46.319186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:46.321460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:46.323707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 252/490] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:46.325459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0391e-2262-4d0c-8fc8-7476bd263f99 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.325859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0391e-2262-4d0c-8fc8-7476bd263f99 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.326224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0391e-2262-4d0c-8fc8-7476bd263f99 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.329819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 253/491] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:46.330924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0391e-2262-4d0c-8fc8-7476bd263f99 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.360063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:46.426037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1650e-e774-484e-bf88-28c287b83615 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.442486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:46.470424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.471757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.472432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.473160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.474685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:46.475435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.476682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0391e-2262-4d0c-8fc8-7476bd263f99 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.479631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7edb7-4451-44c0-9eb2-4cace2f236a6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.496126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 239/492] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:46.498789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.499467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.500077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.519825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.520606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.520987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.521364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.522637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:46.522637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.524951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.531555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:46.533623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:46.567336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 254/493] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:46.568268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.663665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 240/494] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:46.666859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.714487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d82fa-31f8-48c4-842c-2053736dfe88 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.752785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26632310-0709-460a-9990-efd72257f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.753339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632310-0709-460a-9990-efd72257f75d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.753617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632310-0709-460a-9990-efd72257f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.753970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26632310-0709-460a-9990-efd72257f75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.756130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:46.759826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:46.772028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26632310-0709-460a-9990-efd72257f75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.781355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26632310-0709-460a-9990-efd72257f75d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-26632310-0709-460a-9990-efd72257f75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:46.806907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:46.817772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 255/495] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c408-d123-44ae-bc2c-7b6cbf906f99 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c408-d123-44ae-bc2c-7b6cbf906f99 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c408-d123-44ae-bc2c-7b6cbf906f99 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 256/496] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:46.818620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f85cd-6267-4681-9565-0d284880648a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.819142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f85cd-6267-4681-9565-0d284880648a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.819142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f85cd-6267-4681-9565-0d284880648a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.819885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 257/497] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:46.822420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f85cd-6267-4681-9565-0d284880648a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:46.839827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb754e01-15b6-46a5-a9bc-462e23f52dda None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.879540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.880162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.880438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.880708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.942708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f85cd-6267-4681-9565-0d284880648a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:46.962897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.964605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29625b57-4816-4348-9dce-0fb28811133f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:46.987370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:46.988035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:46.988342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:46.988590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:46.989249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:46.989547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:46.991058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c24b-3b1f-4ca0-8ec5-464e2bf466c0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:47.005440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 258/498] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:47.006788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.007072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.007341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.019206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.027748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:47.028956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:47.083151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.087650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 241/499] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:47.087650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.087650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.087650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.108332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:47.109542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:47.116377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 259/500] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:47.117230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.191833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 242/501] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:47.193474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.278422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c690683-c167-4e88-8e7e-9fd09e5c2293 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.339299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.353090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.354586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.355294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.355974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.376982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28b33-edfc-4a96-b6cb-e0aad71c16fd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.380695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:47.389539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:47.391135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:47.392600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:47.396398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 260/502] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:47.397354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.458066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.458726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.459001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.459262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.468173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:47.469625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:47.470914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:47.483669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:47.485542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 243/503] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:47.487651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.601183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb7ca6-280b-4068-bf63-a05003cdc5a6 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.625202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4646d-62ee-4dbd-8a24-91428f655710 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.639892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.640638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.640905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.641170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.652091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:47.653541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:47.661395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.662033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.662317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.662584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.663240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:47.663540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:47.665531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:47.668466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:47.669352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:47.670524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:47.670524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:47.670524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:47.670524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 261/504] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:47.670830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.671147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.677827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 244/505] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:47.678905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.773098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.787530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164d617-8c74-4e91-916a-2e8a6c24c968 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.797639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480763d2-eb4b-4d50-89c0-0c347566ba81 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:47.825115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.825786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.826154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.826351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.835868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:47.837792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:47.838079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:47.838336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:47.838520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:47.843460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:47.844528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 262/506] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:47.851003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.854348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:47.855787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:47.875823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:47.876995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:47.878079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:47.878079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:47.878079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:47.878079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 245/507] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:47.878079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:47.943474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:47.964798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c81f3-4fa8-4bbb-917c-ef5a3e943b26 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.002843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12e7ff-fc99-4c1c-ac06-b5fbfed4afda None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.004081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.004692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.004963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.005229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.005991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:48.006292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.007732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.030676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 263/508] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:48.031719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.048408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.048673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.049173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.049454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.049758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.050436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:48.050738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.052145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.066543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 246/509] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:48.067473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.170144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3072d-aa22-4193-bc8b-8275322f4144 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.183544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c800b1-74e6-42ec-9aef-98fc2f73d444 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.199065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.214629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.220914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.220914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.220914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.224521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.225137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.225412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.225743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.226392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:48.226690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.227911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.228846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.229373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.238481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 247/510] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:48.240839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:48.250954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:48.251972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:48.252949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:48.252949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.252949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:48.252949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 264/511] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:48.253235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.317782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.363022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31f0-109f-4f42-b8df-3b7cbca79a0d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.365440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e01-39c9-41f7-a900-d5359ce0618f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.398911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798c650-2859-4b45-9c91-95475694318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.399780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798c650-2859-4b45-9c91-95475694318f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.400056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798c650-2859-4b45-9c91-95475694318f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.400327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798c650-2859-4b45-9c91-95475694318f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.400969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798c650-2859-4b45-9c91-95475694318f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:48.401270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798c650-2859-4b45-9c91-95475694318f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.402803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798c650-2859-4b45-9c91-95475694318f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.404032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.404671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.404967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.405241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.429026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 248/512] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:48.429026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798c650-2859-4b45-9c91-95475694318f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.465865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.487083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.488694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.539650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798c650-2859-4b45-9c91-95475694318f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.550080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.575061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.575711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.575987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.576263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.583544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.585027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.597542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 265/513] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:48.598161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:48.599013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:48.600166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:48.600747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.600871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 249/514] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:48.600871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.716494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.724512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bce6c4-aa1d-4205-8c1b-f849450b0976 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.730032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889f948-2893-4c29-8e33-8b538a43eb18 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.769050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.769667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.769991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.770261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.770911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:48.771209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.772603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.775766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.776477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.776936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.777046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.785609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 250/515] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:48.786478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.791537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.793098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:48.810650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:48.811973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:48.812811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:48.812811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.812811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:48.812811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 266/516] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:48.813034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:48.827131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:48.907002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baebadd-0e73-40f5-9d45-7ad574c0f3b9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.926626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437546aa-3835-40b8-b219-1b4253398160 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:48.943266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.943911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.944300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.944493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.945146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.947335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.967748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:48.968298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:48.968570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:48.968836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:48.980007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:48.981463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:48.996728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:48.998180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:48.999246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:48.999246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:48.999246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:48.999246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 267/517] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:48.999246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.061938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.105003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 251/518] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:02:48 2022] POST /identity/v3/users => generated 247 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:02:49.105987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.133775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53a2d5-7281-4cec-9caf-cd2ad84d9022 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.137849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.197280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.198277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.198696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.199113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.200277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:49.200753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.203475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.242300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 268/519] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:49.242300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.300864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39974b7b-a6b6-406d-b89f-687334a5e9d6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.347127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.347773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.348141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.348311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.356828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.359064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:49.372799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:49.374383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:02:49.377305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:49.377999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28be16-f1b3-4c50-88d6-f1ac8ca9e721 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.378215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 252/520] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:49.378215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.406179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.418362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.419174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.419595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.420084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.420944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:49.421267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.422787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.437374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 269/521] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:49.443164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.468005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.514134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310596ba-abdb-4344-85f8-fae5c65aa774 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.560802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.561428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.561772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.562032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.562661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:49.562963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.564386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.567716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e841-2d5a-4309-b0d7-c2158ff44795 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.592060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 253/522] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:49.592060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.632684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.634508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.634508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.634780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.635520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:49.635825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.637339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.653824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 270/523] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:49.654882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.737747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933914-81d4-49df-aaf5-fe9b2feea3c5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.756029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.796377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:49.807819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.809297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.810138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.814268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.815476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:49.815792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.818885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.865745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc3e19-cdeb-4e9f-9c95-79ef9b7d5e18 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:49.872847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 254/524] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:49.873750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:49.913500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:49.914932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:49.915342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:49.915710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:49.924002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:49.925699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:49.950174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:49.951165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:49.952164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 271/525] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:49.952722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.023008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7189f50-16a3-4aad-a1c4-5dbd719c4122 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.076381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.077358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.077897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.078364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.081835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f22178-1e63-47cd-a5df-0ae0da10fe34 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.090700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.092410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.095287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:50.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:50.118413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:50.119343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:50.125812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 255/526] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:50.127662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.132621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.134093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.134696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.135152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.217182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.218950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.230124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.310164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19ad10-e4f3-4871-9c47-6eeeed86499d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.352577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.353255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.353504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.353792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.354531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:50.354844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.356549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.377755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 256/527] 10.222.0.99 () {56 vars in 1139 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:50.379023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.388132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 272/528] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:50.389138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.403219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.499682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff8bdd-cdbb-4f32-81ff-73547bb045a6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.509662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.518298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e194cab-ddf1-40bc-a47c-8e75874c4985 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.540703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.541569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.542017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.542477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.543352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:50.543840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.545832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.566535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.567300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.567580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.567850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.572569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 257/529] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:50.577573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.680771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.682443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.690768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.710787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b532d-dc43-4679-8abf-56b39d2e45f0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.750933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.752452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.752928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.753396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.765127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.766946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:50.783947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:50.784915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:50.785829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:50.785829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:50.785829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:50.788094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 258/530] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:50.789780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.799114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 273/531] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:50.800081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.813632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.906496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422a20-d94d-4a9a-ae36-95a727044489 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.924298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d536a-d3fb-44b6-bc71-69fd7610d888 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:50.933188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:50.944370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.945132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.947084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.947472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.949636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=be3a2d2b02f44ce9bd64ffd38675b6d4) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:50.950103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.951676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:50.960638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 259/532] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/role_assignments?user.id=be3a2d2b02f44ce9bd64ffd38675b6d4&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:50.962547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:50.974465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:50.975098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:50.975434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:50.975707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:50.976367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:50.976669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:50.978382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.003567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 274/533] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:51.012196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.174226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:51.190956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a573284-805e-4091-aa42-2ba55d257239 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.243450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.244963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.245525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.246123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.260528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.262573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.272473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e1f2-33dd-4066-a51e-b3643f839c9c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:51.282536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:51.283554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:51.284422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:51.284422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:51.284422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:51.286922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 260/534] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:51.294823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.347735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.348401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.348685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.348958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.357473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.358972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ecfde4d1-3def-498c-9515-b70fcc39f3c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:51.373220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:51.374393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:51.375438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 275/535] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:51.375908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cea38-2841-46da-8ad9-73b48890b8ff None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.376246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cea38-2841-46da-8ad9-73b48890b8ff None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.380853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cea38-2841-46da-8ad9-73b48890b8ff None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.382674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 276/536] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:51.384825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cea38-2841-46da-8ad9-73b48890b8ff None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.431946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:51.468581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0262-e385-416e-bd2f-bce5668675d1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.514000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.514682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.515000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.515274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.515992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:51.516320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.517903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15ba94-a884-4875-9430-bf1b4ec8b920 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.519993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:51.529553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cea38-2841-46da-8ad9-73b48890b8ff None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.532955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 261/537] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:51.534417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.534704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.534982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.556655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:51.557978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:51.576531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.577150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.577588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.577915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.578921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:51.579225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.580766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.605843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 277/538] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:51.607522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.643857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 262/539] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:51.644792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.770108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:51.780532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e891a2-0da9-47b7-9db7-a4310f288804 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.787733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebadd9-9fb8-4412-9c05-9af46c32d5a2 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:51.807827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:51.828010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.828721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.828998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.830694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.830694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=ba2b8b21f0454436a1958cb3abb27e8f) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:51.830694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.831901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504d8f0-68b3-46e4-bf9b-6d9f8b3eff7b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.839654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 263/540] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/role_assignments?user.id=ba2b8b21f0454436a1958cb3abb27e8f&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:51.845231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a03e1-9d2e-49a5-b8ad-a8b185087ade None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.845920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a03e1-9d2e-49a5-b8ad-a8b185087ade None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.846419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a03e1-9d2e-49a5-b8ad-a8b185087ade None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.848376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 264/541] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:02:51.853019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:51.853743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.854055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.854287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.855677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:51.856220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:51.856785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:51.871180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:51.872975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:51.880146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:51.881821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:51.899162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:51.900133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:51.901070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 278/542] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:51.906834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:51.975250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 265/543] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:02:51.981471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.084752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab077a77-b578-4abb-ab9f-6e02c7fec28d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.123264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:52.124306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:52.124765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:52.125221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:52.126234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:52.126541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:52.127998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:52.142374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 279/544] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:52.149876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.162119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:52.197487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c097a8-4673-4f74-8901-ccd4abbb839a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.242094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:52.293163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:52.295053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:52.295506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:52.296122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:52.317442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:52.334857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:52.335946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 266/545] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:52.337766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.385091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da219-0edd-4df0-83cc-28f8e33d0584 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.431629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:52.432359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:52.432729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:52.433181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:52.434066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=e1564eaa8d8543faa415c09ddbdf4c01) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:52.434442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:52.435995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:52.442948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 280/546] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:52 2022] GET /identity/v3/role_assignments?user.id=e1564eaa8d8543faa415c09ddbdf4c01&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:52.443894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.532065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:52.571425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d05f-13f0-48e7-adb4-32facfd1338c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.617969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:52.617969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:52.618245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:52.618341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:52.619508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:52.619508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:52.622652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:52.639829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 281/547] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:52.643697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf28a2-1736-432d-8888-d360f7041c83 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.651531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.708089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:52.708734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:52.709023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:52.709291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:52.734756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:52.742550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:52.744166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:52.745335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 267/548] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:52.763580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:52.844029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:52.936040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853fb4-106d-4b2f-a96d-f8d6a37d815b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:52.936886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af681376-3d5f-4b9e-81fc-fa27d3f06f47 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.002396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.007312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.008660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.009335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.009631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.010344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:53.010649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.012136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.015702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.016410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.018690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.018690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.035896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 268/549] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:53.037957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.040152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.043707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:53.070925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:53.071944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:53.072976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:53.072976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:53.072976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:53.080468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 282/550] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:53.089513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.209996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.221085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a444745-a405-4657-9e8f-4594c9df5721 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.228442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5b86a-ce7d-4375-a7a1-3c1ee44a5c88 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.291570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.292346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.292689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.293035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.309794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.311565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.312954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.314408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.315244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.315936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.317405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:53.318532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:53.330276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:53.332395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:53.335209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:53.335209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:53.335209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:53.336122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.343457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 269/551] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:53.354813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.364513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 283/552] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:53.372666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.443817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.505006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5580c8-8f76-4913-8f65-bdd5f08b240c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.522735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817544cc-16f1-4ed5-8c70-d9d183bf5782 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.561981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.563020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.563450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.563874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.569791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.571373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.571871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.572359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.573564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:53.573943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.574162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.576190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.576994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcced-2a97-47ef-8b84-6297116861d9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:53.593825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:53.595906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:02:53.598188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:53.607923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 270/553] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:53.607923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71522a-3ecf-4943-a554-a4d77b629dc0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.607923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71522a-3ecf-4943-a554-a4d77b629dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.607923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71522a-3ecf-4943-a554-a4d77b629dc0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.607923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 271/554] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:53.610351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 284/555] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:53.620121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.623747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.625347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.628273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.652538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:53.654154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:53.666362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.736735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 272/556] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:53.737895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.846623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24701ded-6057-4eb0-b49e-c53a402997bc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.853266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0bc0c-e462-41af-8bd3-60680d709cbc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:53.858736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.885579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.886370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.886697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.887027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.887856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:53.888223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.889822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:53.905564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:53.915486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 273/557] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:53.916303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:53.954996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:53.956527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:53.957280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:53.958043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:53.984187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:53.987352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:54.026756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:54.028280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:02:54.029312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:54.041700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 285/558] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:54.046845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48478e0-b4be-448a-ac2b-6326b74c8d12 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:54.063366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.111419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:54.112776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.113537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.114640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.245929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:54.251242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:54.253104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083668-9052-4a58-b780-5ac8422f295f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:54.253927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:54.330045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:54.332247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.332984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.334061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.335094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:54.335619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:54.339396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:54.385906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 286/559] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:02:54.387505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.432051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:54.457812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 274/560] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:02:54.459470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.547966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f912536-a7ab-45d5-bd56-7a7f11c1239c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:54.591454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:54.592560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.593057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.593558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.612149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:54.619900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:54.623993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 287/561] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:54.624817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.645957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:54.650953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb43a1-4717-43e1-9a5f-bf5f27eccab0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:54.709459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:54.710608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.711897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.712250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.730893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:54.733745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:54.740153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-df4c99f0-89a5-4612-832d-ded29c005f8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:54.757039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:54.759439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:54.760900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:54.760900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:54.760900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:54.764615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 275/562] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:54.765547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522b83-be42-4e63-9213-8931a4b8af6d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.765864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522b83-be42-4e63-9213-8931a4b8af6d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.766131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522b83-be42-4e63-9213-8931a4b8af6d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.769851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 276/563] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:54.769851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9522b83-be42-4e63-9213-8931a4b8af6d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.781296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946fb2-ec35-4bc5-ada2-f615065606b7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:54.831901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:54.832530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:54.832811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:54.833077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:54.845311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:54.846824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:54.865028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:54.867622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:54.870004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:54.870004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:54.870004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:54.870004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 288/564] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:54.870004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:54.962869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9522b83-be42-4e63-9213-8931a4b8af6d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.011821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.012779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.013212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.013684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.014589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:55.015024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.016928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311d736-ec8a-4307-8b7c-2aad82e61bbe None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.035843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 277/565] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:55.035843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.035843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.035843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.042266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762bf71-fcf6-404f-8125-4a262fb63a21 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.054424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:55.055901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:55.066136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.078510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.079099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.079372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.079642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.084150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:55.084150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.084150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.084150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.095665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 289/566] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:55.096522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.129152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 278/567] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:55.130432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.223971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48007433-fccb-4bd8-bd6f-82d4ed1e88ed None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.285110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.285110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.285110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.285110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.285697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:55.285697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.285697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc87eba-05fc-4952-be31-5ad3c2620169 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.288714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7408a4-1ce8-4fd5-9d8c-d55912d45a55 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.305764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 290/568] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:55.305764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2fe68-0e8e-454d-88d4-d3b5a39432e5 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.305764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2fe68-0e8e-454d-88d4-d3b5a39432e5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.305764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2fe68-0e8e-454d-88d4-d3b5a39432e5 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.309590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 291/569] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:55.310626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2fe68-0e8e-454d-88d4-d3b5a39432e5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.350405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.352553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.353189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.353589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.374300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.377025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.381959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.386528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f18898e3-58cb-4b68-ab92-4be0f3d297d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:55.407074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:55.416878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 279/570] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.418693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.444645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:55.446796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:55.451386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2fe68-0e8e-454d-88d4-d3b5a39432e5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.487246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.487802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.488083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.488373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.496966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.498478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:55.510986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:55.512692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 292/571] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:55.514443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.550203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 280/572] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:55.551070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.678063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b299ec-3dc7-42f8-83e4-4014c11d796f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.691163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f86747-c455-4236-b40f-c1fe51e9321d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.699989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.728354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:55.739731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.740690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.741193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.741690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.742497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.743587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.744069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.744513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.754498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.756118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.764575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:55.769086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:55.770962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 293/573] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:55.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:55.776085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:55.779295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:55.782634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:55.785386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 281/574] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:55.785386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.920292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6b4b-996f-4c00-a819-5fa9e7bbbfae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.947367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd07d7-e165-4f24-b11a-02fe1a327d35 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:55.964386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.965286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.965286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.965574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.966222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:55.966523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:55.968728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:55.983010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 282/575] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:55.983882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:55.995479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:55.996527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:55.997165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:55.997932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:55.998784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:55.999151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.003519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.026525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 294/576] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:56.027386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.045007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.063833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.114039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e790b-60d4-4e80-b3ee-f2d8ce8cfd21 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.150931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73980698-f1ca-4a38-b63f-023cf236ce3d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.156082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.157019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.157458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.157940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.159084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:56.159576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.162137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.186695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 283/577] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:56.187550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.204459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.205079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.205357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.205628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.206323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:56.206625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.208014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.217351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 295/578] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:56.218231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.311084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448f31b-ef8c-44e7-94ae-b8577395a559 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.319169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.339767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.345616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e375a-acfa-4e11-a8b8-d45183ea723c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.363235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.363861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.364146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.364415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.378509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:02:56.383418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:56.384401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:56.385240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 284/579] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:56.385964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.386073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.386554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.387007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.387481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.411599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=2d95be82e9e04470902aef0d9bab1919, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.414677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.502041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e54fd-7cc6-43d6-8594-66fa2a920205 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.503137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 296/580] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:56 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:56.504112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.548889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.549758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.550053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.550353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.560600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.562276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:56.576023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:56.577387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:02:56.578811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:56.582714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 285/581] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:56.583730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.632096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.711824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372d359-8cbc-491a-a7ae-4c25228a7cef None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.740790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bce72-c132-4c7e-91ee-71b10efe3f43 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.761756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.762734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.763179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.763913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.764987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=4e95d57ba23a424e8ce7e7e1695cd8a8) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:56.765515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.768057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.778313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 297/582] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/role_assignments?user.id=4e95d57ba23a424e8ce7e7e1695cd8a8&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:56.779624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.798495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:56.804646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.805970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.806419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.806939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.825647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.828238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:56.856867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:56.857945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:56.862751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:56.862751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:56.862751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:56.862751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 286/583] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:56.862751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:56.929621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919dfde-f44b-4108-b409-6e26fbcf657b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:56.980940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:56.982220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:56.982677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:56.983115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:56.984234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:56.984728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:56.987262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.011551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 298/584] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:57.013091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.022782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:57.061222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff2e97-a2ae-4a63-9ef9-522082ba6473 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.099378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.099909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.100188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.100453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.101283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:57.101637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.103235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.115601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:57.126049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 287/585] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:02:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:57.127249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.163509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557110b-a460-476d-91b2-f5a305d31e4b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.225820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.227571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.228117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.228548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.229979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:57.230526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.233496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.266353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 299/586] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:57.268124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.287948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7090c-1677-4d42-89bb-3dbf553c951a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.336217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.337204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.338046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.339610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.343995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:57.359025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.359025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:57.374099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:57.375103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:57.375963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:57.375963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:57.375963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:57.379902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 288/587] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:57.380844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.428268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c58d78-f3b0-4295-881c-17ccc4f35725 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.471675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:57.479520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.480848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.481339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.481847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.495248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.497347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:57.516946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:57.518801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:57.520386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:57.520386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:57.520386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:57.521902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 300/588] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:57.523591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.554190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f572c0f-7293-4a8d-8c22-7d416f88634f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.597817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.598464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.598746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.599018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.618966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.620457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:57.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:57.641065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 289/589] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:57.643083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.708958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:57.756414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a977fbc-6fbb-411b-b453-ba94829aa2ee None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.777412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef57ec-39f4-4754-b561-1a3d8d2e2676 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:57.828910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.830335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.831007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.831706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.833026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:57.833883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.837307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.844639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:57.845737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:57.846262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:57.846770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:57.848069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:57.848642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:57.851583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:57.863026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 301/590] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:57.864887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.885809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 290/591] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:57.886789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:57.935862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.001253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c79f5-fac4-4422-94a0-53d0ed44e57f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.008836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ff905-45b9-4030-9b20-e19cf850cd32 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.019269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.040551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.041491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.041959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.042372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.046463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:58.046463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.046463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.046765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.047282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.047387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.047592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.055824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.057346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.067690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 302/592] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:58.068800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:02:58.075402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:58.076508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:58.077517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 291/593] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:58.083058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.183428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c230-28e5-40f1-97fb-fc60e7686156 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.185427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.197045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e39cf-6a69-437a-ab71-b2bf9d6b166c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.222409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.223365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.223811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.224262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.232723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.234539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.240196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.240831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.241110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.241385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.254323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:58.255547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:58.256406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:58.257289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:58.258285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.265264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 303/594] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:58.266571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:58.273170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:58.274899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:58.275771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:58.275771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:58.275771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:58.275771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 292/595] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:02:58.275999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.307173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.391489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c937-f918-4e84-9f16-89a4b293955b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.393910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e02b97-1f37-4ac2-835f-8de988130edd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.432455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.433082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.433363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.433703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.434368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:58.434674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.436125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.438091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.439077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.439551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.440012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.441056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:58.441613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.443471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.455014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 293/596] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:58.455014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.473860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 304/597] 10.222.0.99 () {56 vars in 1139 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:58.476676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.507440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.563663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.588316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4202b1-379b-4040-a689-bcda0d0721ba None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.621275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732d859-1184-42ff-902d-4a31b7d56414 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.645107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.646135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.647180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.647664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.649532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:58.650340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.652694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.662061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.662723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.662997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.663268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/be3a2d2b02f44ce9bd64ffd38675b6d4/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.671635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 294/598] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:58.672843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.689444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=be3a2d2b02f44ce9bd64ffd38675b6d4, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.694216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.793765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 305/599] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:58 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/be3a2d2b02f44ce9bd64ffd38675b6d4/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:58.793765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:58.825469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70ab0f-ccc5-4f54-81f6-a19486de373b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:58.911237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:58.913047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:58.914452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:58.915669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:58.934583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:58.940937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:58.944745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:02:58.978226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:58.979911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:02:58.981486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:02:58.981486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:02:58.981486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:02:58.991713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 295/600] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:58.994323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.003980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:59.047502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9470e7-c1d2-412e-a0e2-5a1ef50f06e4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.104894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.105434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.105632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.105931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.116074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.119101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-52a26ab0-a3c0-418e-8844-02eb7fc732de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:59.136246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:59.139724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 306/601] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fa8-084a-4c5e-bee6-cf5952cf6e26 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.142007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fa8-084a-4c5e-bee6-cf5952cf6e26 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.146020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fa8-084a-4c5e-bee6-cf5952cf6e26 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.146020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 307/602] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:59.151172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac376fa8-084a-4c5e-bee6-cf5952cf6e26 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.203236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964b52-8942-40cc-970d-8aa51f7985fb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.250381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.251657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.252266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.252916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.254257 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:59.254972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.258255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43773bd8-5a7d-4542-a5da-c8643e60ad46 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.283730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 296/603] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:59.286214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.286912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.287494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.319293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:59.323076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:59.337593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac376fa8-084a-4c5e-bee6-cf5952cf6e26 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.381057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:59.389662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.390536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.390954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.391364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.392701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:59.393171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.395797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.410358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 308/604] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:59.411478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.426076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:59.478002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 297/605] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:59.478002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.639177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a4893-bf08-4608-aa75-604330418d93 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.653094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199ba05-9b16-449f-afd7-e70686f0e8e7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.692861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.693874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.694895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.696241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/ba2b8b21f0454436a1958cb3abb27e8f/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.732565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.733471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.735316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.735747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.737207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=ba2b8b21f0454436a1958cb3abb27e8f, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3572e-98b6-4e39-b1e3-49ce64659809 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.751586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.753359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-479ec495-35ad-43b6-96a4-4fc7c2250561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:02:59.767340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:02:59.769395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 309/606] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4fe4-81f5-439d-9952-e7e448719240 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.771841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4fe4-81f5-439d-9952-e7e448719240 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.774822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d4fe4-81f5-439d-9952-e7e448719240 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.778223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 310/607] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:02:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:02:59.778223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d4fe4-81f5-439d-9952-e7e448719240 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.841584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:59.852099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 298/608] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:59 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/ba2b8b21f0454436a1958cb3abb27e8f/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:02:59.853510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.853813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.854080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.877702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:02:59.879269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:02:59.921770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:02:59.927561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d4fe4-81f5-439d-9952-e7e448719240 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:02:59.968090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:02:59.968722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:02:59.969005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:02:59.969278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:02:59.970215 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:02:59.970498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:02:59.972060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:02:59.972968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 299/609] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:02:59.974097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:02:59.995797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 311/610] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:02:59.995797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.118989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf113-5bfd-4f48-8e35-d824203acaf5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.126617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbe164-c6bf-4f96-8735-d0c367775a27 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.148045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:00.173917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.174860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.175139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.175706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.187636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.188790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.189464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.190096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/e1564eaa8d8543faa415c09ddbdf4c01/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.196833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:00.208356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:00.210034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:00.211781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:00.213477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 300/611] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:02:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:00.230217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.232263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=e1564eaa8d8543faa415c09ddbdf4c01, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:00.235116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:00.272572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:00.332261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 312/612] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:02:59 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/e1564eaa8d8543faa415c09ddbdf4c01/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:03:00.375332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.376910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60855-a870-4e20-b0b9-8e3fe06e7390 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.426738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.429513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.429973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.430233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.430956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:00.432730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:00.437793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:00.462300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 301/613] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:00.467943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.591509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3201-9f37-4a61-9093-f14a42cd5181 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.650553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:00.660241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.666318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.666318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.666318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.668096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940b954-b593-40c2-98f9-155d8f5e1316 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.680181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:00.690966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:00.692945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:00.694748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:00.699562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 313/614] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:00.705407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.706623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.707483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.708554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.709371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.728891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:00.731905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:00.737095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:00.766051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:00.767067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:00.767970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:00.767970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:00.767970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:00.773900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 302/615] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:00.780907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:00.849653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21caeb-2a87-40f7-84cf-bf3c446dcbfa None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:00.889314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:00.890443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:00.890929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:00.891484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:00.892419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:00.892967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:00.894592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:00.906339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 314/616] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:00.916456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.028469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557e8f9-c340-4bfd-a8f1-1964fd12f57d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.090068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.090684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.090958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.091243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.091862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:01.092156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.093520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.096299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:01.112224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 303/617] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:01.120357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.134591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:01.172023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a951a26-752e-4385-aa1e-bc633b9f9f7e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.281677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.282425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.282690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.282959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.325639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.327660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:01.372597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:01.373587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:01.374843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:01.374843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:01.374843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:01.376860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 315/618] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:01.382156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.388202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ffb1e-e6e5-4dcf-a454-84ec23b1de2f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.468764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.469786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.469786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.470069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.494536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.496061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.526896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7b41f-722a-4ce1-839a-0dd5c01b58a7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-7f5910eb-b3ab-4ef2-856f-8993712f54c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:01.527670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:01.528619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:01.529550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:01.534775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 304/619] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:01.537198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78929b7-1db7-4e79-ba10-a4b4149b6f7b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.537692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78929b7-1db7-4e79-ba10-a4b4149b6f7b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.542143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78929b7-1db7-4e79-ba10-a4b4149b6f7b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.545336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 305/620] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:01.547550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78929b7-1db7-4e79-ba10-a4b4149b6f7b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.575328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.576321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.576807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.577281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.578196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:01.578740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.580386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144208-7f87-4fa5-a225-ed3ff1635a01 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.582434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:01.595459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 316/621] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:01.597104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.597576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.598425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.619091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:01.620738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:01.632182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:01.695925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78929b7-1db7-4e79-ba10-a4b4149b6f7b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.701734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 317/622] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:01.703040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.769385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.770478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.770975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.771546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.773812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:01.774525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.786918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.819640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157110e8-0a33-42cf-9a7e-14e6c2db2168 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:01.827384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 306/623] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:01.837100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:01.847399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:01.867148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:01.870703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:01.871210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:01.871945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:01.894697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:01.897674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:01.920254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:01.921280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:01.922258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:01.927958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 318/624] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:01.934392 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.056787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010fce-8510-43dc-95d8-0467a8becf95 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.076267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.104887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.106217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.106704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.107362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.131665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724e7b-c2da-46ba-aaf4-3a9ae8fd89f0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.182430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.184008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.184482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.185198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.186297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:02.187029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.189459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.201808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.203587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.216529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 319/625] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:02.217683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.270032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.324349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 307/626] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:03:02.325373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.333868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9264e-47a7-48bf-9e0a-48aeeec15e44 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.342135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.374205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.375181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.375644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.376092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.389838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:02.395516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:02.396438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:02.397559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:02.400570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 320/627] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:02.401655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.451040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df71b7-9f2a-43e8-9617-96bb2b38fc17 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.515730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.516361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.516644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.516914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.524701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6720a0-311c-47ab-932c-dfa1277216ad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.527125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.528303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.529692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6c3c9cb5-ab6a-40d8-ac1b-8e379dc1703e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:02.549651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:02.550880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:02.551946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:02.551946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:02.551946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:02.554744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 308/628] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:02.556558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503999f-b68e-4052-9480-58b5e94cd973 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.556855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503999f-b68e-4052-9480-58b5e94cd973 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.557130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503999f-b68e-4052-9480-58b5e94cd973 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.559127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 309/629] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:02.559970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503999f-b68e-4052-9480-58b5e94cd973 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.577339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.578015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.578298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.578573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.590242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.591881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:02.608950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:02.610242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:02.612087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:02.612087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:02.612087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:02.612881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 321/630] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:02.613232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.666288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.707847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503999f-b68e-4052-9480-58b5e94cd973 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.754712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.755896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.756187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.756459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.757131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:02.758020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.759863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b1164-32d7-4bc7-8da7-6999dcfc780a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.777681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 310/631] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:02.777681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.777681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.777681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.779149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc6b7f-b194-485b-ae38-aac794ee43e4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:02.794038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:02.795226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:02.842496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df970-24db-4969-b800-846265779a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:02.843104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df970-24db-4969-b800-846265779a76 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:02.843383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df970-24db-4969-b800-846265779a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:02.843649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df970-24db-4969-b800-846265779a76 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:02.844327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9df970-24db-4969-b800-846265779a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:02.844629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9df970-24db-4969-b800-846265779a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:02.846047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9df970-24db-4969-b800-846265779a76 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:02.859663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 322/632] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:02.860387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df970-24db-4969-b800-846265779a76 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.897741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 311/633] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:02.897741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:02.898644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:02.986234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.042760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df970-24db-4969-b800-846265779a76 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.046560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c5441-902c-47e0-a69e-338098589e25 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.092391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.093719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.094240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.094739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.097358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.097993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.098459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.098736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.099376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:03.099931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.101134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.114896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.116381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.118286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 323/634] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:03.118419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f8627047-af78-4889-bb33-097aebaca9a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:03.148272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:03.150976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 312/635] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670a144-7815-4ba6-9a1e-bd36bda7db40 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670a144-7815-4ba6-9a1e-bd36bda7db40 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670a144-7815-4ba6-9a1e-bd36bda7db40 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 313/636] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:03.153876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670a144-7815-4ba6-9a1e-bd36bda7db40 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.209820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.235571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbdb1-0364-4ddc-8de1-614f582136ad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.289789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670a144-7815-4ba6-9a1e-bd36bda7db40 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.290736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.291413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.291694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.292023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.299539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.301023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.305727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-f63b5952-b4a8-424a-a277-61420e52a492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:03.317652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:03.322130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 324/637] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.323002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.323648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.333823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.334464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.334736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.335005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.342831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.343006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:03.344226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:03.344338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:03.358100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:03.360069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 314/638] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:03.362014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.431539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 325/639] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:03.432542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.506505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.507266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bf444-ab27-46e0-a9f8-96b0b0d29352 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.555703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e0d22-1643-4d0d-8a51-ca375dd2b3d2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.567630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.568240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.568524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.568795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.569440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:03.570299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.571682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.588421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 315/640] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:03.588421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.625835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.633044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.634151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.634581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.635058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.647552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:03.653501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:03.658520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 326/641] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:03.660339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.737394 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364aca7-9744-4bfb-b61c-404aee962ef4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.781774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.782404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.782683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.782945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.783681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:03.783988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.785443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.812203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 316/642] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:03.812203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.812592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727605-d419-428f-ae13-a22d896dcacd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.841728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.853547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.858846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.873520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 327/643] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:03.874664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:03.916740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:03.932153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2ccb3-9932-4bc4-8641-f42a98a35e41 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:03.968734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:03.969368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:03.969640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:03.969947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:03.978338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:03.979810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:03.991562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:03.999274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 317/644] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:04.001547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.003004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0385-24c0-4e62-a334-b98e59273c11 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.037211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.037854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.038135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.038433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.039678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:04.040012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.041437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.050617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 328/645] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:04.051453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.104261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.117644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808095-55ef-4890-b345-3174a6122d2c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.155670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.156303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.156577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.156845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.160874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc4d36-26ac-4d84-a768-c46a072cc64b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.167485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:04.172710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:04.174044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 318/646] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:04.174989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.187579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.207414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.208564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.209057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.209555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/4e95d57ba23a424e8ce7e7e1695cd8a8/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.234604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=4e95d57ba23a424e8ce7e7e1695cd8a8, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.236056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.283743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bddb86-8014-4038-971d-19a8f7b986d1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.323322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 329/647] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:04 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/4e95d57ba23a424e8ce7e7e1695cd8a8/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:03:04.324206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.343711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.345490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.345490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.345490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.346039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:04.346272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.347783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.373845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 319/648] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:04.373845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.494690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.505794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7e62-61fb-4027-9c79-138c461ccce9 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.543865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.558858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.559837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.560161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.560479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.561139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:04.561446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.562992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.579108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c13e03-5982-43c2-99b0-4588ce290a13 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.590645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 330/649] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:04.591731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.641820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.642357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.642634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.642910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.657411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.658935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:04.680601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:04.682600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:04.685239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:04.685239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:04.685239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:04.685239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 320/650] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:04.685239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.812285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20915e-bc9e-40ea-99ee-e74d9b8bb581 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.871237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.872532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.873023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.873555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:04.874504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:04.875041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:04.876672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:04.883409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462ab63-d1fa-4cd3-b8a2-e61143119139 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:04.890416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 331/651] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:04.891681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:04.917154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.939337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:04.958134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:04.960000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:04.972040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:04.972040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.028383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604ae13-e9c3-4a61-8620-09fe5ae8f7e6 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.080573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.082822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.084188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.085948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.086496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.087043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.088808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:05.088808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.091624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.120127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 332/652] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:05.121301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.249295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 321/653] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:03:05.250457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.251592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.283559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511246d-165c-406d-bb2c-53a9bc735e9e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.290263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.328454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.329452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.330015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.330471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.343031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.344837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:05.361128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:05.362337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:05.363246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:05.363246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:05.363246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:05.365459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 333/654] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:05.367171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.392523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a97cd6-4c6f-4735-91a6-a0654ee56908 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.442954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.443600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.445781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.460218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.460218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.460218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:05.472120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:05.473131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:03:05.475454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:05.478221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 322/655] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:05.479138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.525582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4365945-11b1-4bc2-92fe-51906c75c7f7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.557587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.566956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.571299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.571299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.571299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.576186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.583511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.585412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.592450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a4361-82f0-4aa4-b8ce-79e92104bc9d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:05.606755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:05.607761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:05.608642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:05.608642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:05.608642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:05.610895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 334/656] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:05.611949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.643084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.643701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.644483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.645244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.646582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:05.646996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.648777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.662706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 323/657] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:05.664401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.743386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b425238-8f61-4903-a164-89b6d133cd1a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.783182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.783910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.784288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.784665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.785510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:05.785968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.787704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.803686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe60f08-3464-4fae-9526-54a50777c78f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.811433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 335/658] 10.222.0.99 () {56 vars in 1137 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:05.812574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.855463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.856246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.856633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.857008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.857835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:05.858341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:05.860016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:05.866134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.875306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 324/659] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:05.876313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:05.886899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:05.947296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1b7-390b-4cb3-b345-87d24a8f4b3e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:05.994880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:05.995792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:05.996216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:05.996898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:05.998545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b393-2162-45b7-a65d-1f1443fc7d19 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.005344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.007198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:06.020134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:06.021127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 21:03:06.022088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:06.025181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 336/660] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:06.026244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.038414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.039466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.039752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.040029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.121257 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.123453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.131917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.136861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.141451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb563-92c0-440a-97e4-f76a804a9c8b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.178482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.180012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.180456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.180884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.189019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.190857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:06.205031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:06.206080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:06.206968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:06.209670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 337/661] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:06.211560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.237296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 325/662] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:03:06.237928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.328756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62788-c291-4587-be40-13eeb387506f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.362745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7a72c-5dbe-4873-975b-f1f04e2c69e1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.369010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.369955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.370397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.370826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.376124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.376248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:06.376248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.376248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.395001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 338/663] 10.222.0.99 () {56 vars in 1139 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:06.396206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.402813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.403435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.403714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.403983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.404683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:06.404981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.406588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.418621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.421354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 326/664] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:06.422261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.513682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed62e59-1460-4189-b0bf-5d5347a66c6f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.540051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93481756-b578-4298-b80a-ea85784c1ed7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.551437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.552410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.553257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.553798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.566270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.568197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.586169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.586864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.587088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.587365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:06.587895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:06.588790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:06.589667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:06.589667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:06.589667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:06.591482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 339/665] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:06.592529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.659118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.693223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:06.706604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.708190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f850ea-0da8-43cd-8b8e-c86307c15e73 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.747047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059d799-9a4c-472f-968f-0b849c1da514 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:06.788831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:06.789051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.789411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.789646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.790356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=be3a2d2b02f44ce9bd64ffd38675b6d4) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:06.790661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:06.792106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:06.805764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 340/666] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/role_assignments?user.id=be3a2d2b02f44ce9bd64ffd38675b6d4&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:06.805764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.857993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 327/667] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:03:06.859387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70cf6-63bc-4443-83c7-48b183839481 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:06.859746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70cf6-63bc-4443-83c7-48b183839481 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:06.860112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70cf6-63bc-4443-83c7-48b183839481 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:06.865774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 328/668] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:06.866866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70cf6-63bc-4443-83c7-48b183839481 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:06.978034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.000129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc4e70-dfc2-4670-9be3-7b60c2a380e8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.042636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70cf6-63bc-4443-83c7-48b183839481 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.062399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.070059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.070798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.071443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.071732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.072817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:07.074199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.075266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb8206-7cd4-4fc9-940b-8465b6bfcdcb None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.092367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 341/669] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:07.093595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.093914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.094188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.104034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.105112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.105389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.105657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.114881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:07.115992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:07.119724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.122285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:07.139853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:07.140841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:07.141739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:07.141739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:07.141739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:07.141739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 329/670] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:07.142400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.196784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 342/671] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:07.197683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.321401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53a9-b327-4e0a-acb0-9b4daaa527c6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.341677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.369112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06311-99b5-4920-a562-09ab8dca41e3 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.376810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.378154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.378675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.379180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.396741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.398530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.400197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.411274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.411979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.415266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.415266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-bbee00a0-4c40-4298-ae79-15814def5683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:07.415572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:07.416734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:07.419579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:07.421282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:07.421282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.421282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f7b5e-efa3-48cd-83b1-8caf3f9cc938 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.421991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 330/672] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:07.421991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21b1b-cb98-4d7a-bce1-bfc39de61e4a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.421991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21b1b-cb98-4d7a-bce1-bfc39de61e4a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.421991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21b1b-cb98-4d7a-bce1-bfc39de61e4a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.421991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 331/673] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:07.425768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d21b1b-cb98-4d7a-bce1-bfc39de61e4a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.428951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 343/674] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:07.430474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.430728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.430998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.451648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:07.452854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:07.531412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 344/675] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:07.532087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.557384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d21b1b-cb98-4d7a-bce1-bfc39de61e4a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.615066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.616871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.617995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.618447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.619360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:07.619816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.621851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.644567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 332/676] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:07.645548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.669678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c262-40d6-4729-b14d-e1cca6aaeab6 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.688068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.706739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:07.726743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.727825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.728403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.728964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.744020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.748404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:07.775217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:07.776189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:07.777083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:07.778099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 345/677] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:07.779665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.788126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e658-c534-44d6-b3fc-41c28d69d02e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.831188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.832271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.832621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.832959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.846969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:07.852399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:07.854366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:07.856164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:07.858463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 333/678] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:07.860672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:07.940141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55b87a-34f5-4618-aebc-790290f9a773 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:07.982365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:07.982971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:07.983252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:07.983528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:07.984244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=ba2b8b21f0454436a1958cb3abb27e8f) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:07.984546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:07.986001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:07.987574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.005753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 346/679] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/role_assignments?user.id=ba2b8b21f0454436a1958cb3abb27e8f&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.005753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.006100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1018b0d-c101-499c-8a20-b30377ff146a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.057137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.067745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.068646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.068943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.069324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.070713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.071088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.072732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.087380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 334/680] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.088336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.171928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260897c-764d-4ce5-b1e4-7d0a407cd39e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.215135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.215843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.216236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.216577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.230184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:08.235877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:08.236890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:08.238069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 347/681] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:08.238780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984652b-e7de-43a0-a55b-082aaa1f862c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.238857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.299593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.300594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.301374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.302213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.303085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.303573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.307048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.325769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 335/682] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.327248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.359893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.376129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.384154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43353b67-91bf-415d-849b-bb345a82e218 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.426604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.427739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.428248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.428812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.430564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=e1564eaa8d8543faa415c09ddbdf4c01) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.431225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.433047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.445847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 348/683] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/role_assignments?user.id=e1564eaa8d8543faa415c09ddbdf4c01&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.447177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.452193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf0348-0428-49b6-b6d6-322ddb188d32 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.502408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.502949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.503232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.503507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.504183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.504635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.506146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.528064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 336/684] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.531303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.606561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a117c6-117a-4294-a576-4cae803636ad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.634793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.650274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.650921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.658045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.658045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.658235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9b43a-bbea-41f8-a482-a942cf015dfc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.665533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.667125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:08.686938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:08.687917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:08.688777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:08.688777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:08.688777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:08.692986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 349/685] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:08.694277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.708262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.708908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.709215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.709482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.713025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.723416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.724888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:08.741460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:08.742502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:08.743378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:08.743378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:08.743378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:08.743378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 337/686] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:08.750260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.808389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed0592-2cee-442f-8a6f-6ff8385c797a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.850019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.850688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.851262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.851949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.852610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.852913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.854336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.873594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 350/687] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:08.875005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.879487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ef3c9-7a82-4384-aa5f-1a19556ce0cf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:08.918869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:08.919754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:08.920229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:08.921380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:08.922307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:08.922772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:08.924409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:08.935299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.939923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 338/688] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:08.945701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:08.957931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:08.995989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893c96d-99d6-4bec-92c8-e5310d39dc0d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.040826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.041499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.043002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.043002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.052442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.055331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:09.068970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:09.070572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:09.071484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:09.076318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 351/689] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:09.077815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:09.088561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6d74-4f00-4590-83e4-2e47eec5ca90 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.134906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.135913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.137582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.138093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.151041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.157626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:09.174840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:09.175833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:09.176755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:09.193023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 339/690] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:09.195149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:09.279126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:09.293760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dd19c-522a-4863-83c1-fb13f91356be None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.328837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7dd04-8afe-4464-939c-2b0df850d716 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.374962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.375635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:09.376815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.377307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.377841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.378872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:09.379418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.381073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.394505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.395133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.395417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.395773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.396544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:09.396849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.399414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.405856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 340/691] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:09.412012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:09.444322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 352/692] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:03:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:09.454814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:09.545867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c573fc0-d097-43dd-a152-7c95067ff227 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.561525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:09.585783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.586950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.587333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.587797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.617057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343558f-7e2c-4fdc-8ff7-41e74ecfe963 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:09.666906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:09.668096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.668555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.669015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.690998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.693206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be6a80-6f2b-4f8d-989e-e1566e34a7d3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.752668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:09.768525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:09.770330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a579e7-29d7-46c5-805f-2a8bf2f26586 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:09.826732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 341/693] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 21:03:09.828333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c7cd-1d56-4227-a0e3-9d703d16ccb0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.828756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c7cd-1d56-4227-a0e3-9d703d16ccb0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.829096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948c7cd-1d56-4227-a0e3-9d703d16ccb0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.834902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 342/694] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:09.836377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c7cd-1d56-4227-a0e3-9d703d16ccb0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:09.854535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:09.917156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 353/695] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 21:03:09.918833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:09.919316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:09.919816 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:09.945063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:09.946880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:09.970272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948c7cd-1d56-4227-a0e3-9d703d16ccb0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.011865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.012639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.012987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.013340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.027205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.028847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.044969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 354/696] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:10.046251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:10.054348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:10.055510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:10.056451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:10.056451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.056451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:10.056451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 343/697] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:10.061326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.075675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.190680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.227266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab781b-74ec-4d4e-ab59-f4ff39ffe1a3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.241120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafd637-e098-4d09-8460-872de2180a93 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.279797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.280988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.281548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.282222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.300768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.301869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.302379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.305019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.305922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:10.306442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.308848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.314664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.316800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.334177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 344/698] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:10.340503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:10.357655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:10.359491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:10.361360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:10.361360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.361360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:10.366876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 355/699] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:10.373771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.463406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.478379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6405a-280e-4bfd-8155-8ea6c4b7c2f4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.496059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87f04d-e21e-4337-9bd7-c8035e904a7f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.504555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.516794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.517662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.520028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.520028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.525696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.527229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.533940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.534558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.534832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.535111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.535760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:10.536061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.537480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e619bf-9d94-4305-a0d0-f4ff072052d0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:10.538864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:10.539760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 345/700] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:10.541107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.554426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 356/701] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:10.555697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed15-a3c1-42f0-83b5-277aa6f8f79b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.555979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed15-a3c1-42f0-83b5-277aa6f8f79b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.556251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04eed15-a3c1-42f0-83b5-277aa6f8f79b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.561616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 357/702] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:10.561616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eed15-a3c1-42f0-83b5-277aa6f8f79b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.669321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2bb8c-8198-4850-ac51-bb2ca50d7f7a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.679727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04eed15-a3c1-42f0-83b5-277aa6f8f79b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.705172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.705836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.706164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.706439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.713302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.718645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.719092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.719708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.719982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.720090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.720288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.720939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:10.721240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.722842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.740258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 358/703] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-e7db590c-1ee2-44d7-99dd-c45ae7576b5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:10.740478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:10.741409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:10.743148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:10.743148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:10.743148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:10.743319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.743397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 346/704] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:10.743397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.743397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.743397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.761706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:10.762968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:10.794780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:10.835691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 347/705] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:10.836533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.883369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8600188-24fd-43ee-84bf-3359f6daf6b2 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:10.921246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:10.922095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:10.922510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:10.922877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:10.930536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:10.932167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:10.945622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:10.946719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:10.947792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:10.954583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 359/706] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:10.955641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:10.978017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a362d-39fe-4ce0-9028-b2de3cdc7c9f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.018331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.027015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.027674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.027947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.028211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.028928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:11.029298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.030734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.045915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 348/707] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:11.045915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.087700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.090801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55bfa9-036f-4fcc-b514-a3bdc56a1b09 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.143092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.143728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.144109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.144283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.144928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:11.145232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.146667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33990166-86cf-4002-ad77-fca43746bcaa None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.162145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 360/708] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:11.163903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adacc-a094-4c98-9c69-afa84eb0fa88 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.164204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adacc-a094-4c98-9c69-afa84eb0fa88 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.164477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adacc-a094-4c98-9c69-afa84eb0fa88 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.166081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 361/709] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:11.166827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adacc-a094-4c98-9c69-afa84eb0fa88 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.209251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f8ddb-1eb2-471c-855c-25da4abf5c7f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.258385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.259102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.260037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.260454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.261300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:11.262135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.264462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.277078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 349/710] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:11.278209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.294062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adacc-a094-4c98-9c69-afa84eb0fa88 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.309964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.334377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.335308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.335911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.335911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.336260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.337642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-2429deac-da79-4174-a15f-6a0d1cd4fd01 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:11.349365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:11.350817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.352381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 362/711] 10.222.0.99 () {56 vars in 1129 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:11.353186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.353186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.353186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.375649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:11.377641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:11.389315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.399737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32acbea-3801-427f-8eb5-c3bbedb6df91 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.439313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.439938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.440195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.440463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.441131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.442591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.452031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 363/712] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:11.452692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.566262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 350/713] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:03:11 2022] POST /identity/v3/users => generated 254 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:11.566949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e79f5-a330-43e8-baee-3825b5fff0f2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.567172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.576856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.620844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45e20-50ab-496b-8891-540f16295def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.620844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45e20-50ab-496b-8891-540f16295def None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.620844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45e20-50ab-496b-8891-540f16295def None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.620844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45e20-50ab-496b-8891-540f16295def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.621544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45e20-50ab-496b-8891-540f16295def None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=4e95d57ba23a424e8ce7e7e1695cd8a8) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:11.621544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45e20-50ab-496b-8891-540f16295def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.621544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45e20-50ab-496b-8891-540f16295def None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.628920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 364/714] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/role_assignments?user.id=4e95d57ba23a424e8ce7e7e1695cd8a8&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:11.755827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45e20-50ab-496b-8891-540f16295def None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.816417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abaaba-7cf2-420f-854a-acf73ebd47dd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.859881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.860850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.861335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.862727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.863582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:11.864098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.865736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.871645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 351/715] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:11.877356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:11.892627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:11.901245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45e20-50ab-496b-8891-540f16295def None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:11.961281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:11.961789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:11.962064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:11.962333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] PATH_INFO: `/v3/services/network` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:11.963027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:11.964415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:11.974859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:11.976089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:11.976957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 365/716] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:11.982803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:12.034062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:12.035502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b1d03-910a-4716-9735-7075462bbf9b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:12.077010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:12.077546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:12.077852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:12.078119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:12.078758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:12.079053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:12.080444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:12.088914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 352/717] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:12.126743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da12493-4d23-4520-afba-660f08f1548d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:12.167362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:12.168342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:12.168810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:12.169261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:12.170141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] RBAC: Adding query filter params (name=network) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:12.170668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:12.172376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91f8d3-581a-4057-9b54-3181f890aaae None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:12.178260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 366/718] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:12.185781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:12.320838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:12.370797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968d8dc-79a3-45b0-9250-56825b230a78 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:12.412587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:12.414179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:12.414465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:12.414738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:12.415487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:12.415794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:12.417183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5176c-d4f5-4799-bc5b-e1a18d99c2e1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:12.422709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 353/719] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:13.399204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519238-654f-4140-b03a-e4413253e145 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:13.399780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519238-654f-4140-b03a-e4413253e145 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:13.400040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519238-654f-4140-b03a-e4413253e145 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:13.402031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 367/720] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:13.406991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73115b-ea2a-4d35-bd25-9323f5a8d29d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:13.407493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73115b-ea2a-4d35-bd25-9323f5a8d29d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:13.407589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73115b-ea2a-4d35-bd25-9323f5a8d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:13.434802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ea73115b-ea2a-4d35-bd25-9323f5a8d29d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:13.436373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ea73115b-ea2a-4d35-bd25-9323f5a8d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:13.541597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 354/721] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:13.982689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519238-654f-4140-b03a-e4413253e145 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.119180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:14.121740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbedec1-ba77-4e01-af6a-65fd97a0ad38 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.122023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbedec1-ba77-4e01-af6a-65fd97a0ad38 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.122295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbedec1-ba77-4e01-af6a-65fd97a0ad38 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.124262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519238-654f-4140-b03a-e4413253e145 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:14.125360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 355/722] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:14.129634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.130011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.130229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.150498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:14.151701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:14.189527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:14.190385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.190659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.190933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.203557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:14.208440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:14.209484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:14.211240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 368/723] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:14.218705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.296177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:14.303953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 356/724] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 21:03:14.384755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176ba96-1c0e-4d42-a5e3-81d6d87d9cee None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:14.421680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:14.422528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.422650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.422921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.423495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:14.423792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:14.425241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6649a8f-a4b1-466c-9530-849bb3a692ed None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:14.435899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 369/725] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:14.443473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.600417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd441c-7176-40d0-845b-8404793adc62 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:14.626653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edcd4-79f1-47c8-88ad-62dcb0815575 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.626653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edcd4-79f1-47c8-88ad-62dcb0815575 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.626653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3edcd4-79f1-47c8-88ad-62dcb0815575 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.627683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 370/726] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:14.634182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.634524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.634742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.672262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:14.673704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:14.674045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.674324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.674591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.676259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:14.697548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:14.699370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:14.707408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:14.732136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:14.733177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:14.734116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:14.734116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:14.734116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:14.739401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 357/727] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:14.745396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.798771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 371/728] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:14.800031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.821288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:14.883302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896559-98e0-47e1-a2ac-8efcd8a3aa22 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:14.922934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:14.923576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.923876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.924125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.937099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-cba397f5-fec5-463f-8788-832380bea758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:14.950285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:14.951644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:14.952502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 358/729] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:14.953218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd843082-2464-49c7-9e62-8cbaf9cc39c8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:14.953829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd843082-2464-49c7-9e62-8cbaf9cc39c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:14.953829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd843082-2464-49c7-9e62-8cbaf9cc39c8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:14.956025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 359/730] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:14.959272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd843082-2464-49c7-9e62-8cbaf9cc39c8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:14.989134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95d02a-44c9-4ac9-820b-9fc4deb6fd87 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.037141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.037857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.038282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.038562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.039216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:15.039511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.040887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2c76c-3f92-4cf7-80f8-847beb9a2e04 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.052654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 372/731] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:15.054850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.054996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.055150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.073585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:15.074806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:15.077673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.092778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd843082-2464-49c7-9e62-8cbaf9cc39c8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.105199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.130072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.130679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.130949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.131214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.132488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:15.132786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.134238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6070c0-349c-4c5e-937d-944fb50dec02 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.142902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 360/732] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:15.144202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1be8d6-ccb6-43ac-8010-5822081bcc09 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.144729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1be8d6-ccb6-43ac-8010-5822081bcc09 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.145046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1be8d6-ccb6-43ac-8010-5822081bcc09 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.146719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 361/733] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:15.147682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1be8d6-ccb6-43ac-8010-5822081bcc09 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.157636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 373/734] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:15.158532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.262433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1be8d6-ccb6-43ac-8010-5822081bcc09 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.270399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df88cac-0460-46bc-9b22-c62b70b57307 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.307060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.308254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.308829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.309187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.310223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.310817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.311093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.311417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.321805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.322036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.323992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.325801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:15.332398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:15.342177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 374/735] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:15.343333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-2060c3b4-c78e-4c99-b3c5-6039fd94c2d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:15.344184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:15.345604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 362/736] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.349079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.351572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.364552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:15.365967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:15.396366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.446424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 363/737] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 21:03:15.447412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.467178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d19303-c2ed-4b5a-ae38-e5a4d5f46296 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.514012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.514666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.514918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.515233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.527844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.529308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:15.545368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:15.546411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:15.547426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:15.547426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:15.547426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:15.547426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 375/738] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:15.547426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.566729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c366a-8948-4e35-8c45-0e747a7dfaa5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.607246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.608125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.608494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.608893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.609531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:15.609856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.611241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.611890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.620902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 364/739] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:15.622017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.658430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.693778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6631fc4-050d-474d-9f54-8f6c3dcd96f4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.735837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.736457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.736740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.737011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.737771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:15.738082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.739518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.741673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8aedda-8f1c-4e11-a41f-1316a0501975 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.758708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 376/740] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:15.759836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.779963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.780566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.780839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.781107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.781784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.783136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.858479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee474-e48e-437f-9f25-45c4017e34bd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:15.869940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:15.893960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:15.894615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:15.894897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:15.895171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:15.895810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:15.896113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:15.897481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:15.909102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 377/741] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:15.909874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:15.923216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 365/742] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:15 2022] POST /identity/v3/services => generated 251 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:15.924069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.026975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.061124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad596-20be-4255-b0a0-0e2cff8a38f6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.102207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.102855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.103134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.103407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.104080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.105490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.145511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18473217-5b5e-49c7-a1ca-5a7e652e8aee None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.181606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.182252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.182524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.182796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.197852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.199283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 366/743] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:15 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:16.199404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.200078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:16.223699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:16.228513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:16.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:16.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:16.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:16.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 378/744] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:16.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.269619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.360215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b399-a265-4682-a723-97d000cc4071 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.369099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e834b01-6c96-4f80-b2fa-c9c363aada32 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.401656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.402338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.402614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.402905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.405931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.414795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.414795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.414795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.415748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.415826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.416983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.428218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.430455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:16.434301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:16.435830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:16.438148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:16.438148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:16.438148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:16.438148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 367/745] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:16.438148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:16.446159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:16.449852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 379/746] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:16.456280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.558299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.570503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9dfbc-848d-42cc-a010-49d8e036332c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.583381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573c66a-3bce-4d74-857f-a0babe829df6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.622017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.622637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.622916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.623184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.623907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:16.624209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.625606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.630991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.633181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.633734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.634211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.635079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:16.635580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.637196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.639092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 380/747] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:16.640067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.652803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 368/748] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:16.654036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.709309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.749111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13540cc-3365-4fed-97b2-6611f85c6180 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.770243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e492582-3db3-4522-be31-c73bb80d97b4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.787034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.787729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.788646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.788917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.789652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:16.790024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.791473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.811218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 381/749] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:16.811795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.812027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.812675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.812951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.813250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.813915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.821843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.823302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:16.836749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:16.837778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:16.838671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 369/750] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:16.839171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:16.927663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4c973-f3c4-409d-a58c-c7efaf59871c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.954752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a888df-b37f-45bf-a1dd-5c70fbb571f8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:16.967095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:16.967705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.968586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.969104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.969590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.978158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:16.979920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:16.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:16.994171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:16.995118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:16.997993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 382/751] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:16.998201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:16.998858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:16.999141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:16.999413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:16.999871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.064336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:17.083772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.086521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.111725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7806ad-e68b-4f63-aff7-0322e21a3338 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.151275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.152267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.152754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.153229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.154195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:17.154703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.156369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.170141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 383/752] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:17.171321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.205356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 370/753] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:17.206338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.219517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:17.291626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9e797-3af6-4fe3-b7f3-d41811356e89 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.314397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:17.330088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.330847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.331183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.331520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.332318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:17.332685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.333626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ba66e-4e74-4114-a33c-94686a9230b7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.334281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.354068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 384/754] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:03:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:17.355084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.375601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.376559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.377000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.377421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.379715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=2d95be82e9e04470902aef0d9bab1919, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.381328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.538047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 371/755] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:17 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:03:17.539564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.560624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed490-93e3-490c-90d3-4d0ec54b0d16 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.603918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.605013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.605516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.606017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.620181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.622321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:17.641285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:17.642408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:17.643354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:17.643354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:17.643354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:17.645573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 385/756] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:17.651511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.690097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:17.715007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:17.772946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70085d2b-626e-4a5b-a135-1a03a2bff958 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.809603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.810380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.810557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.810807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.824512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56531ae9-a21c-4c5b-82bd-b064f0ba8204 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:17.862755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:17.863769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:17.864257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:17.864754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:17.865625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:17.866354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.867949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.882210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 386/757] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:17.887540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:17.888537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:17.890081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:17.974103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.009338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 372/758] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:18.014855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.077129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.089705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cafd1-d2f4-417d-a356-68c506dff454 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.137095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.138032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.138714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.139164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.149775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.152216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.162390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e7196-0826-4dcc-91cc-3832206fbc81 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-75c70450-3908-4a03-817c-ebfd5a01d269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:18.170629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:18.171775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:18.172844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:18.174061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 387/759] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:18.175359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0a03-cd00-4855-8168-f03210f1ab0b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.175730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0a03-cd00-4855-8168-f03210f1ab0b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.176074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa0a03-cd00-4855-8168-f03210f1ab0b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.177669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 388/760] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:18.180827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa0a03-cd00-4855-8168-f03210f1ab0b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.222916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.224345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.225106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.225806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.245338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.248507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8bc88dd4-0ce7-4618-8b28-e1691ca875ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:18.276931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:18.279103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:18.281177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:18.281177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:18.281177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:18.284107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 373/761] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:18.286410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.287067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.287834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.318322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:18.321332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:18.348410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.365377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa0a03-cd00-4855-8168-f03210f1ab0b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.428956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.430137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.430701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.431134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.432651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:18.433148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.435967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.439052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 374/762] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:18.439052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.454460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 389/763] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:18.460735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.474772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.575549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4bcaf-2c93-4395-a3db-f2012b75efd7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.616987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.616987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.616987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.616987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.617456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=b8c976a6a3bf48a6a518bdd7b604087a) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:18.617811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.619374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7799064-d9a8-49f2-ba70-0450d690e9f2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.626090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 390/764] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/role_assignments?user.id=b8c976a6a3bf48a6a518bdd7b604087a&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:18.635480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec83f3-a245-4dbc-810b-017dd29b4ba1 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.635870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec83f3-a245-4dbc-810b-017dd29b4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.636249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec83f3-a245-4dbc-810b-017dd29b4ba1 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.636466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122e33c-ee3d-4df0-8095-78da8e140658 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.637805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 391/765] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:18.643984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.644310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.644593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.681886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:18.683129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:18.717987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.718828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.719018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.719237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.719910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:18.720212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.721645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.737409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 375/766] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:18.738482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.766131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.797898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 392/767] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:18.799452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:18.809289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:18.890648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf70df7-3da5-4852-a812-f80b7e3930c4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.922836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169702c-8770-4b11-956c-3668b14e7416 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:18.942162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:18.945462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:18.945676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:18.945986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:18.946766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:18.948314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:18.977345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:18.978766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:18.979684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:18.980454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 376/768] 10.222.0.99 () {56 vars in 1129 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:18.989304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.014395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.015148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.015430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.015702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.032287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.033690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:19.059739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:19.060765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:19.061676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 393/769] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:19.073701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.097738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.132612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76e298-443f-4882-8c82-feb1b7598aea None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.238117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.280655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca68-604c-4ab8-bd35-a4b8a26fc40f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.308645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.309273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.309548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.309849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.310838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.311146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.312604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.320599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 377/770] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.322165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.327379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.328394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.328872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.329340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.330288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.330798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.332600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.348853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 394/771] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.350822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.481426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca515f-78f8-4dc3-a57d-ce5c863dcb35 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.507398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f01412-84b0-4665-b765-2460acea459b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.541393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.545275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.546766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.547530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.548369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] PATH_INFO: `/v3/services/network` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.549889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.554612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.554782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.554782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.554782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.554782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.555296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.555296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.556553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.564073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 395/772] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.565008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:19.572910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:19.574719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:19.576377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:19.582800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 378/773] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:19.585156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.590879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.693009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b3968-ba22-46c3-980b-a665aa46ac60 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.697934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9792af-a8ba-4dce-9b00-04efb650225d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.737611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.738418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.738795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.739194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.739994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.740125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] RBAC: Adding query filter params (interface=public, service_id=bbc8b98a049647cda0b388ecf96755a3, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.740608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.741080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.741504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.741931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.742586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.742965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=4e95d57ba23a424e8ce7e7e1695cd8a8) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.743414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.745207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.752654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 396/774] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/role_assignments?user.id=4e95d57ba23a424e8ce7e7e1695cd8a8&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.753757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.759072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 379/775] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/endpoints?service_id=bbc8b98a049647cda0b388ecf96755a3&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.760318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:19.830042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.839553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:19.882039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af507f-cbc3-4864-a328-88d347155224 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.906887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f552-8529-40fa-9fd3-188c2e30fc6b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:19.926454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.927227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.927439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.927709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.928404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.929835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce8406-afea-433f-acab-5a487dcd5ef2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.936553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 380/776] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.937971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8060-d93e-44ba-852b-9361f6dc366b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.938253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8060-d93e-44ba-852b-9361f6dc366b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.938523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a8060-d93e-44ba-852b-9361f6dc366b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.945364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 381/777] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:19.947231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.947615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.947990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.964997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:19.966006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:19.966437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:19.966856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:19.967665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] RBAC: Adding query filter params (name=network) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:19.968117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:19.969762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:19.970037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:19.971605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:19.977057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 397/778] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:19.984396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.070992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 382/779] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:20.129917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02b0ee-345c-4034-8528-83bfc058826e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:20.150960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:20.173271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:20.173935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:20.174214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:20.174486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:20.175297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] RBAC: Adding query filter params (type=network) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:20.175599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:20.177090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:20.183633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 398/780] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:20.191248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.290925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:20.346219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0962395-ee37-4704-9dc7-1293f38bcdc9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:20.399123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:20.399758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:20.400039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:20.400315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:20.401030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] RBAC: Adding query filter params (interface=public, service_id=a04326dda3fd4f1e8f2bb80d60cdb364, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:20.401427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:20.402665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.402858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:20.409023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 383/781] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:20 2022] GET /identity/v3/endpoints?service_id=a04326dda3fd4f1e8f2bb80d60cdb364&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:20.415642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.526139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa33f-fac2-4eec-a685-caea9ef9e76e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:20.554353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:20.573670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:20.574325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:20.574600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:20.574871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:20.575481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:20.579792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:20.587420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 384/782] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:20 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:20.654976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573ee67-8431-43db-a99c-a3da7bc34c24 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:20.692782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:20.693438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:20.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:20.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:20.706921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:20.713031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:20.714013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:20.714845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 399/783] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:20.722926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.870062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814b682-3417-4848-a7af-5660e9e02567 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:20.913282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:20.917769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:20.929868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 385/784] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:20.935138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:20.986808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:21.052850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32d3b-1205-4880-bbb6-149a07e91ba4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:21.106137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:21.106781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.107063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.107344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.119837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:21.121640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-db573d3f-d44a-4083-abec-a7c5508bd908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:21.151222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:21.152209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:21.153098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:21.153098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:21.153098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:21.153098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 400/785] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:21.159594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:21.214075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8a8c8-de85-4867-b837-703778324d5b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.214462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8a8c8-de85-4867-b837-703778324d5b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.214629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8a8c8-de85-4867-b837-703778324d5b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.216104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 401/786] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:21.221358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.221637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.221924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.253173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:21.253867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:21.254695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:21.358282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738b07b-246d-4dc1-89c2-7b9e7773ab31 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:21.396847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 402/787] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:21.426494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:21.431833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:21.448646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 386/788] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:21.454384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:21.489055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:21.598773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032001b-c0cc-49ea-bf1c-03bba83b96df None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:21.661473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:21.662084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.662405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.662683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.670990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:21.672448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:21.691196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:21.692197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:21.693118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:21.696215 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 403/789] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:21.702982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:21.710389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:21.848595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:21.871803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233893e-3ead-445c-8424-6b520c0f2c1e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:21.917803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:21.918345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:21.918625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:21.918917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:21.942240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb281ab-5e6f-45e4-97c0-19e603b1f3ca None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:21.946889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:21.951787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:21.952714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:21.953528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 404/790] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:21.962791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:21.967715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.028177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.029052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.029470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.029932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.030817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:22.031263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.033047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562713a6-c228-456d-ae62-ccbf95cfea39 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:22.072195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 387/791] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:22.078369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04fbd-8700-4a96-9cf2-de91931923e1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.078694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04fbd-8700-4a96-9cf2-de91931923e1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.079060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b04fbd-8700-4a96-9cf2-de91931923e1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.081637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 388/792] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:22.083883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b04fbd-8700-4a96-9cf2-de91931923e1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:22.145588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1937e0-a549-4a71-b31f-23202fc21ce8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:22.190803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.191346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.191631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.191897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.192577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:22.192875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.195099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df6491-ff4c-4d15-ad43-9e3b2409c6b9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:22.207447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 405/793] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:22.208923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.209526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.210063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.218846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.224286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b04fbd-8700-4a96-9cf2-de91931923e1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:22.231822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:22.233198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:22.275474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.276393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.276823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.277492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.332926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 406/794] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:22.332926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:22.335538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.414032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.415503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd0fe7a-4673-408a-b2e1-1ba59c993e4a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:22.537165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3caff-1407-43b0-a792-f73b2e82bbb2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:22.579848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.580494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.580769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.581037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.583244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.585072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 389/795] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:22.586812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77b35a-eb21-42e5-bd59-042f0890d2e1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.587140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77b35a-eb21-42e5-bd59-042f0890d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.587408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77b35a-eb21-42e5-bd59-042f0890d2e1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.593540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 390/796] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:22.594525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77b35a-eb21-42e5-bd59-042f0890d2e1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:22.599690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.601939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-48d2ffec-21bf-4e77-bbf5-68958c6381dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:22.632992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:22.636168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 407/797] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.639030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.668714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:22.670153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:22.701378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.760247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77b35a-eb21-42e5-bd59-042f0890d2e1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:22.781296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 408/798] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:22.781296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:22.811820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.812431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.812716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.812985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.829332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.830925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:22.852756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:22.853895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:22.854891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:22.854891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:22.854891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:22.858180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 391/799] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:22.861291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:22.923008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58a6c8-0574-4b08-abcf-720ac4a0a7c3 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:22.948042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:22.972864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:22.974025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:22.974285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:22.974558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:22.978482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:22.979062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:22.981562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.000008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 409/800] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:23.001246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.016941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aed8f0-344e-4534-aeee-fc0a5956708f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.045703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.065262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.066156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.066422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.066693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.083609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:23.093215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:23.094255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:23.095152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 392/801] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:23.095829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.135187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c77d4-13c6-47a9-99df-35ef6529043e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.181546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.182523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.182979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.183425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.184516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:23.185025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.187473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.212101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 410/802] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:23.212101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.245383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc814f-00a8-46e7-ac42-e1d68909ea20 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.257430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.294302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.295320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.295783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.296239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.307535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.308998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:23.323684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:23.324818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:23.325955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:23.330899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 393/803] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:23.331693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.353413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569184b-aed8-4e42-8027-2f11812b97b0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.395606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.396241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.396515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.396974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.397774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.399236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.399552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:23.409019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:23.420651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:23.422611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 411/804] 10.222.0.99 () {56 vars in 1129 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:23.424442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.468146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42b7f2-1516-4718-b249-09b914eeff0a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.503380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.503987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.504254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.504516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.505172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:23.505466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.507505 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.515942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 394/805] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:23.516652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.529426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.532376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca457379-5d7a-48b9-9b78-25c0d8712fbe None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.566918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.567544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.567815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.568080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.575840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.577206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c3a43780-cc7c-4e1b-920f-076967330e3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:23.589535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:23.590496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:23.591398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 412/806] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:23.591886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563d96-8e97-4461-a523-1da555b5a687 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.592171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563d96-8e97-4461-a523-1da555b5a687 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.592349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563d96-8e97-4461-a523-1da555b5a687 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.593580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 413/807] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:23.594309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df563d96-8e97-4461-a523-1da555b5a687 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.627566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2015599-3be0-4ec6-be07-007cdd2cf1ea None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.629247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.663333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.663915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.664190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.664457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.665177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:23.665475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.666922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.685500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 395/808] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:23.686739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.708872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df563d96-8e97-4461-a523-1da555b5a687 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.751079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.751690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.751965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.752231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.752854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:23.753146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.754551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfda8d-308e-41f4-88fe-f528260a719a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.759300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 414/809] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:23.760503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.760779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.761046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.780178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:23.781443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:23.800022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.808428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947591fc-f031-4baf-acf6-9bc68f4ed296 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:23.847504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:23.848052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:23.848324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:23.848636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:23.855532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 415/810] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:23.856452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.861660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:23.863157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:23.883847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:23.885067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:23.886176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:23.886176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:23.886176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:23.891247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 396/811] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:23.891247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:23.903660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:23.977283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b33bc-2237-42c2-a265-c82d8462ac86 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.010774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef13a-4948-4f08-bf41-63c8c4154969 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.017410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.018049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.018325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.018593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.020302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.020302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.022306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.036361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 416/812] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.037223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.057182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.059027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.059664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.060018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.060211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.072525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.074237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:24.090624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:24.091643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:24.092535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:24.092535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:24.092535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:24.092535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 397/813] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:24.092535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.156777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061f38-90bf-49a5-a592-43977f3c16a8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.177613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.206401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.207527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.207527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.207527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.208145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.208452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.210455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.216275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 417/814] 10.222.0.99 () {56 vars in 1124 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.217105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.220812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfb9c6-b70f-4df6-8809-a9277b6d414c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.267114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.267828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.268105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.268385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.269024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.269317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.270712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.284096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 398/815] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.284947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.317519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.342322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8360525-d96e-4647-be25-ffe34e87c588 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.380812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.382210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.382577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.382855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.383589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.383892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.385352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.391789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 418/816] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.391976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d81b6-30c4-4cad-9eaf-306ef14328b1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.392587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.421542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.434214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.434838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.435112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.435380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.436029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.436328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.437759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.454386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 399/817] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.455888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.539545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4c97-e315-416f-93a1-fb1268d8adc8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.588076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.588948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.589231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.589503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] PATH_INFO: `/v3/services/network` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.590229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.591590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:24.606169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:24.607235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 21:03:24.608093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:24.608844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 419/818] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:24.609555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.617343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac69d-7162-4db1-ad5a-01ec630d035f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.621346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.655907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.656520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.656795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.657063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.657759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.659134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.729273 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00680f9c-b95b-4613-86b4-e58157cdc501 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.767644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.768285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.768546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.768812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.777971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.779413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.782531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 400/819] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:24.783887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:24.810041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:24.811842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 420/820] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:24.812983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.850645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.897150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:24.936565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6fde3-1e7d-4fb2-990e-ce05846a5b34 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.937526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5445b-5a30-440c-9843-b23c37b0e2f8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:24.980560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.981100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.981416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.981520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:24.981747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.982807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:24.982913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] RBAC: Adding query filter params (name=network) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:24.983021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:24.983021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:24.983190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.984143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:24.994599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 421/821] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:24.994599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:24.995695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:24.997537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:25.012410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:25.013402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 401/822] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:25.015027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.126007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1297-5560-473e-b352-ad5c8462855d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.152471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.159082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.163600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef13af-8b90-4075-84c5-bde140cc7a1f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.165579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.166245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.166619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.167740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.167740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:25.168214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.169797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.190359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 422/823] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:03:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:25.191573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.215238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.216157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.216364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.216702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.217407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] RBAC: Adding query filter params (type=network) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:25.217912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.219471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.225428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 402/824] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:25.230812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.310765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6b25c-b8bb-4c1a-86b9-61cedf467e56 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.342531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82308a5-c0e4-49a4-a374-3645531a9a33 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.350951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.351984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.352407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.352836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.353736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:25.354242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.355863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.369478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 423/825] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:25.370686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.381536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.386778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.386778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.386778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.394211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.395665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:25.413521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:25.415942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:25.416835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:25.416835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:25.416835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:25.418893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 403/826] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:25.419792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.436151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.466135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.493132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d82caa-b79a-4d32-83b5-47acb9d1d7f2 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.532216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.533141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.533619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.534325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.535383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.536131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed403e-1f6e-4405-851f-3e6d34115b1a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.537345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.592757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.593404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.593813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.594276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/b8c976a6a3bf48a6a518bdd7b604087a/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.621172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=b8c976a6a3bf48a6a518bdd7b604087a, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.624407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.629700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 424/827] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:25.631639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.748418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 404/828] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:25 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/b8c976a6a3bf48a6a518bdd7b604087a/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:03:25.766337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.770091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5c94a-8bae-4962-bbe3-d7ff824aaad5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:25.812129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:25.813587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:25.814949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:25.815497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:25.816410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:25.816910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:25.818590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:25.833085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 425/829] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:25.843364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:25.939396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:25.996251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3210f37-ba3c-487f-892b-f9779f4868b5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:26.039546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:26.040109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:26.040381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:26.040644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:26.048286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:26.049744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:26.062352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:26.063388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:26.064281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 405/830] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:26.069992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:26.189821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20adcb-cb84-405c-806f-4f33a689f3cc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:26.227966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:26.228581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:26.228855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:26.229117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:26.229880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:26.230112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:26.231612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1ce04-8822-4e45-9ff3-569ae98fc82f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:26.245282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 426/831] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:26.251555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:26.386779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233228a-66d4-40bb-bca5-ee9b2bc8a019 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:26.410209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:26.422006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:26.422668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:26.426964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:26.426964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/4e95d57ba23a424e8ce7e7e1695cd8a8/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:26.426964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=4e95d57ba23a424e8ce7e7e1695cd8a8, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:26.426964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:26.500101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8692e2-03a5-432c-a5d0-b7eff5bcd543 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:26.500490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8692e2-03a5-432c-a5d0-b7eff5bcd543 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:26.500909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8692e2-03a5-432c-a5d0-b7eff5bcd543 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:26.502699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 427/832] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:26.508836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c8a3-4f2f-43a5-8d9d-765c87a5944c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:26.509141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c8a3-4f2f-43a5-8d9d-765c87a5944c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:26.509384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924c8a3-4f2f-43a5-8d9d-765c87a5944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:26.517562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 406/833] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:26 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/4e95d57ba23a424e8ce7e7e1695cd8a8/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:03:26.535005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7924c8a3-4f2f-43a5-8d9d-765c87a5944c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:26.536799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7924c8a3-4f2f-43a5-8d9d-765c87a5944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:26.556879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:26.622549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 428/834] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:26.909501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.026692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dced7-d0b8-4389-a90d-d7fca1783dbd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:27.054668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b02fd-3053-4653-ace2-5e62b4ac2e4b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.055185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b02fd-3053-4653-ace2-5e62b4ac2e4b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.055261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b02fd-3053-4653-ace2-5e62b4ac2e4b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.056877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 429/835] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:27.062049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.062329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.062601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.073458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:27.074517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.075031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.075506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.076294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:27.078030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:27.082827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:27.084141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:27.088920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:27.089918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:27.090832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.097888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 407/836] 10.222.0.99 () {56 vars in 1132 bytes} [Fri Sep 2 21:03:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:27.100330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.153772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:27.167115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 430/837] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:27.223282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51448-6eea-4fa0-bea6-27e135628a48 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:27.265371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:27.266989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.267480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.267978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.268928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:27.269462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:27.271213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:27.278056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 408/838] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:27.284026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.379110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:27.400594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379907ef-b467-474c-8875-da2ed0a0acce None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:27.436065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:27.436933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.437304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.437583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.438296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:27.438599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:27.440028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:27.446059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 431/839] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:27.460534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.609680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343acff-eda6-49db-a4e4-482c4c65ca5a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:27.624071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:27.657362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:27.658435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.659069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.659621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.660240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:27.661678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:27.673587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:27.675853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:27.677802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 409/840] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:27.685632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.838257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbae9d-6b07-438c-8558-bdb9a19fe993 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:27.898660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:27.899820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.899820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.899820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.900218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:27.900404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:27.901808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8d75f-c1a5-4c71-bd11-c81bcfa69837 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:27.911769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 432/841] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:27.913218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:27.932700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b34f-02ba-41f0-9155-9569cd3da469 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.933015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b34f-02ba-41f0-9155-9569cd3da469 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.933500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31b34f-02ba-41f0-9155-9569cd3da469 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.935020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 433/842] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:27.940112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:27.940396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:27.940667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:27.966841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:27.968516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:27.994112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:28.051627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78945-ef08-4318-bd4a-91dfb92f7679 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:28.071900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 434/843] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:28.097751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:28.098612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:28.099032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:28.099967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:28.100755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:28.101209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:28.102805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:28.112337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 410/844] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:28.352714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:28.487325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdff69a-0e69-439d-9a91-ff70e7d78482 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:28.525960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:28.526680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:28.526957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:28.527228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:28.534994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:28.540320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-bb8d82bd-8f27-4bdc-8e19-17f579f8dc94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:28.545374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:28.552132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:28.553134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 435/845] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:03:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:28.553872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:28.637496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:28.715083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba96e3-c073-4b4d-b090-5050bb1c8575 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:28.716292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba96e3-c073-4b4d-b090-5050bb1c8575 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:28.716780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba96e3-c073-4b4d-b090-5050bb1c8575 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:28.726977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 436/846] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:28 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:28.732877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c07f0-0fdf-4f37-b694-40e2917f9cc2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:28.734521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:28.735021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:28.735567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:28.777067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:28.778731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:28.779468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:28.779739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:28.780347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:28.781057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:28.781364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:28.782574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:28.782821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:28.802137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 411/847] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:03:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:28.807684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:28.880656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 437/848] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:28.975632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:28.981703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76a03c-ff01-4fa2-8a4c-44369de176ce None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.029452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.030497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.030971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.031457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.046938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.048729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:29.068671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:29.069871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:29.070746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:29.070746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:29.070746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:29.073255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 412/849] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:29.082918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.196986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a920e-7155-4ec3-992d-398eef1a2475 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.227114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:29.235778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:29.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.241663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.255323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 438/850] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:29.260222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.261317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.402716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d1abe-783c-4540-9769-91de430c12b4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.415359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb705d-b0c6-4d2e-a360-9c51a2beea9c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.454843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.455563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.455674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.455941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.466966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.468443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.468933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.469946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.470741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.471195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:29.482393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:29.483441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:29.484576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:29.488404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:29.489211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 439/851] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:29.491211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:29.497168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:29.498192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:03:29.499121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:29.507163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 413/852] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:29.509360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.528697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:29.571307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:29.638994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dce61-0ad3-4b19-a3df-aa36522b9e2b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.678855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9612626-f228-4402-aedd-825096c57ca5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.683924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.684784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.685207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.685617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.686511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:29.686956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.688560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.711277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 440/853] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:29.719799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.737013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.737628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.737915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.738350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.739153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:29.739594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.742365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.758746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 414/854] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:29.761019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:29.858611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6c64-bd2d-48e1-a594-e8c332156fd0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.868152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:29.873107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:29.901530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9315bae-8304-4169-bad7-df71625a148b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:29.902907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.903732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.904884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.905195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.948555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:29.949544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.950096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:29.950542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:29.968942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:29.971262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-540eb060-6cf9-4216-b02a-7fbbbb9da75f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:29.995310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:29.996573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:29.997582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:29.997582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:29.997582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:29.997582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 415/855] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:29.999122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf3022-16e3-470f-aa09-7c7bbca18c8a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:29.999583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf3022-16e3-470f-aa09-7c7bbca18c8a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.000052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf3022-16e3-470f-aa09-7c7bbca18c8a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.002553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 416/856] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:30.009432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf3022-16e3-470f-aa09-7c7bbca18c8a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.025610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.027857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29779db8-66a3-4987-aa0c-5e19989b2352 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.142844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf3022-16e3-470f-aa09-7c7bbca18c8a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.159024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 441/857] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 21:03:30.160365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0add-4692-4576-b055-57f7533b58b6 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.160718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0add-4692-4576-b055-57f7533b58b6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.160993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd0add-4692-4576-b055-57f7533b58b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.174360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.178534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.180991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e2dd0add-4692-4576-b055-57f7533b58b6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:30.182275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e2dd0add-4692-4576-b055-57f7533b58b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:30.189790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.190609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.191043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.191462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.192244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:30.192543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.194209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.208244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 417/858] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:30.210182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.273166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 442/859] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:30.274221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ec7af-0cbe-419b-b12b-c08557bc0829 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.274506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ec7af-0cbe-419b-b12b-c08557bc0829 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.274779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ec7af-0cbe-419b-b12b-c08557bc0829 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.276042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 443/860] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:30.276753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ec7af-0cbe-419b-b12b-c08557bc0829 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.322393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c63a2-3de0-4f24-a937-95efc1929108 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.362668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.364201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.364485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.364757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.379869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.382809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.399109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ec7af-0cbe-419b-b12b-c08557bc0829 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-cc452537-00fa-426b-8ef2-5b3a9b9e9f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:30.400295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:30.401237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:30.402842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:30.402842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:30.402842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:30.402842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 418/861] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:30.402842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.403140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.403378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.422590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:30.426204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:30.445380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.446348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.446632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.447020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.450591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.455710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.457234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.461344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:30.471307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:30.472283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 444/862] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:30.479539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.509797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 419/863] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:30.511064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.632310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f3936-044b-4f82-92a3-42024107284d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.635186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7dd4-b0b5-4d52-b602-bec2e8b24498 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.671168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.671772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.672052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.672315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.672953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:30.673255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.674655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.675044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.675732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.676080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.676358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.677090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:30.677411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.680043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.690375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 445/864] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:30.691434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.706979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 420/865] 10.222.0.99 () {56 vars in 1135 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:30.706979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.739082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.766317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.813298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501296d-f6f0-4b0b-8408-6badadd760ae None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.822380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207320-9109-4b31-85ad-14638406d209 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:30.852298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.852917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.853251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.853450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.854140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.855516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.859340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:30.859903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:30.860173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:30.860434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:30.868063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:30.869463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:30.880970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:30.882189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 421/866] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:30.883121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:30.970514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:30.986049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec87a-898d-4e3c-a735-d388c461f3d1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.008715 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 446/867] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:30 2022] POST /identity/v3/services => generated 269 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:31.009691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.034790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.035438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.035715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.035997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.129683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.132370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.162537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8a770-f2c1-4f50-a659-85b15ee938f4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.191334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:31.222542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.222840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.223117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.223384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.224047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.226468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.251810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:31.272825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 422/868] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:30 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:31.273825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.329992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 447/869] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:31 2022] POST /identity/v3/services => generated 251 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:31.329992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.392299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d249e-6c50-48e5-9fbf-708b5505671e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.441593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.442399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.442675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.442939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.443847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:31.444186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.446006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.460894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 423/870] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:31.467296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.485065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed8b20-5ade-4557-9b04-b27db4486ef5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.528905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:31.539633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.540297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.541055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.541338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.553922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.555581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:31.574843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:31.575876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:31.576752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:31.576752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:31.576752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:31.576752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 448/871] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:31.582257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.621278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a284c5-9404-4210-9301-06968bcbf0bf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.629922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:31.662964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.663255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.663665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.664014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.664812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=b8c976a6a3bf48a6a518bdd7b604087a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:31.665192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.666771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.673677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 424/872] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:31 2022] GET /identity/v3/role_assignments?user.id=b8c976a6a3bf48a6a518bdd7b604087a&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:31.750352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074bf4f-fcb8-4d95-8173-9ef6b1ca2912 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.811664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:31.813048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:31.813760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:31.814440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:31.815833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:31.816584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:31.819297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:31.846020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 449/873] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:31.851979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:31.982704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf5e1-4bf2-41d0-9af5-dd4c4ae6a073 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:31.985977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:32.022180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:32.022839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:32.023121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:32.023390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:32.035135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:32.037134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:32.050847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:32.051992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:32.053207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:32.055432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 425/874] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:32.061516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:32.157693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:32.170160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154aea97-8457-40d3-a971-464d14ae8480 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:32.208006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:32.208678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:32.209460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:32.209753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:32.210443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:32.210741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:32.212173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18228e32-6621-4fc0-b77f-4e7e2a07bbe9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:32.224909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 450/875] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:32.230783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:32.342893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1abb91-d48c-4688-932b-e4329a95f3dd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:32.378817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:32.381432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:32.381432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:32.381432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:32.381432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=4e95d57ba23a424e8ce7e7e1695cd8a8) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:32.381835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:32.383117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10172b0d-87f8-4ed5-9f0c-8ecebfd1be80 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:32.393801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 426/876] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:32 2022] GET /identity/v3/role_assignments?user.id=4e95d57ba23a424e8ce7e7e1695cd8a8&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:33.104863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a855c-c1be-46c0-907c-a9503210c58d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.105308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a855c-c1be-46c0-907c-a9503210c58d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.105678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a855c-c1be-46c0-907c-a9503210c58d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.107413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 451/877] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:33.112725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2118-4575-4130-b2cb-8d5dfb369b8a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.113203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2118-4575-4130-b2cb-8d5dfb369b8a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.113484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2118-4575-4130-b2cb-8d5dfb369b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.133051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-201d2118-4575-4130-b2cb-8d5dfb369b8a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:33.134284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-201d2118-4575-4130-b2cb-8d5dfb369b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:33.155838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:33.203963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 427/878] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:33.435731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449917d3-5ade-47b6-94d8-494ffdc34af6 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.436230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449917d3-5ade-47b6-94d8-494ffdc34af6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.436230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449917d3-5ade-47b6-94d8-494ffdc34af6 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.437838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 452/879] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:33.443822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.444123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.444450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.465165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:33.467954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:33.503924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449917d3-5ade-47b6-94d8-494ffdc34af6 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:33.553785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 428/880] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:33.643255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449917d3-5ade-47b6-94d8-494ffdc34af6 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:33.679640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:33.680278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.680551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.680818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.681397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:33.682805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:33.692328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:33.693342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:33.694267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:33.697788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 453/881] 10.222.0.99 () {56 vars in 1132 bytes} [Fri Sep 2 21:03:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:33.699650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:33.794158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:33.804521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8ffc2-15e7-4adb-8fb0-47d5447a094a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:33.843658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:33.844281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:33.844552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:33.844819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:33.845458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:33.845778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:33.847161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:33.853335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 429/882] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:33.858952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:33.867991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:33.877596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:33.986729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfceec2-d776-4c48-bd1e-e66deeea2a62 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:33.993153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f60e2e-e8ee-4d3c-9678-53be65dbba4b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.026446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.026972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.027249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.027522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.028153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.028451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.029856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf204d1-1bb9-48eb-adb5-f932cf452371 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.037281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.037933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.038208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.038483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.039066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.040507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 454/883] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:34.040616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.041685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c3c6-dd69-4cf3-a7c7-7f804beb8e13 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.041993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c3c6-dd69-4cf3-a7c7-7f804beb8e13 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.042701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c3c6-dd69-4cf3-a7c7-7f804beb8e13 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.045760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 455/884] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:34.045760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9c3c6-dd69-4cf3-a7c7-7f804beb8e13 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c56d8d40-fa91-4917-8307-b1c08df2aca2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:34.053490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:34.054819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:34.055677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 430/885] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:34.056550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.056550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.056686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.077307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:34.078773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:34.089901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.141613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.152948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 431/886] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:34.153836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.183265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9c3c6-dd69-4cf3-a7c7-7f804beb8e13 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.229819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.230497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.230823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.231024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.231728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] RBAC: Adding query filter params (interface=public, service_id=944229e04bc04cf786cb14aadd7190ba, region_id=RegionOne) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.232028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.233384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.243006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 456/887] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/endpoints?service_id=944229e04bc04cf786cb14aadd7190ba&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:34.248509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.274486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b6371-884d-4018-aed5-1100cd0b118b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.314861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.316048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.316617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.317082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.318131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.318914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.321275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.327980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 432/888] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:34.333190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.345790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.362489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b480bc-4c1e-45ab-be5d-e6fa57b895a0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.409207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.409867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.410147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.410423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.411035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.413250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.425911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 457/889] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:03:34.446871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.451269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.467273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d84b2a-8fdc-4d00-abdb-34b0c3473bda None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.523380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.523945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.524207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.524447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.525093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.525421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.526882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6d99a-2fc8-42e7-abc9-758ab5c465c2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.539382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 433/890] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:34.539382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2909-33b8-499c-92ba-8324ab33d48b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.539382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2909-33b8-499c-92ba-8324ab33d48b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.539382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2909-33b8-499c-92ba-8324ab33d48b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.540226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 434/891] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:34.544015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2909-33b8-499c-92ba-8324ab33d48b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.581956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271eaff6-16eb-4dad-b9b1-185d8bb5bbf5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.633223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.633912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.634186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.634485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.645182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.646464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8bef6451-31fe-4245-bd55-3044446ab0bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:34.652012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:34.652991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:34.654354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:34.658026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 458/892] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:34.659403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.663113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.663113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.688152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:34.690130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:34.703126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2909-33b8-499c-92ba-8324ab33d48b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.746516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.747139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.747413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.747675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.748385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] RBAC: Adding query filter params (interface=public, service_id=c5a3fa73eb0c4a438f2971f548a9068f, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.748688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.750131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.757775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 435/893] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/endpoints?service_id=c5a3fa73eb0c4a438f2971f548a9068f&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:34.758302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.793003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.804401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 459/894] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:34.805342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.903607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc55b0-e483-4d1b-b90c-f02811f83a2f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.943571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.944563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.944935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.945269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.945694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d698a42-8a3e-45d1-94b6-3132ddcfdbd9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:34.946113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:34.946479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.948282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:34.951928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:34.963995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 436/895] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:34.971961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:34.987975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:34.988749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:34.989087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:34.989456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:34.990944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:34.992712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.005898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 460/896] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:03:35.091894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbddc-db6c-4d6f-9d90-5acde9a9e36d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.117324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.135980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.136987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.137451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.137942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.152465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.156858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:35.180374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:35.181415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:35.186819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:35.186819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:35.186819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:35.186819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 437/897] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:35.189796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.198393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:35.262629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:35.267603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d5dd7-6b4a-49c0-84a8-cfac61635297 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.315506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.316670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.317286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.317869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.319238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.320959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.329457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea0b8-5495-4ea7-b9c9-bf4e7931ffaf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:35.330654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:35.331810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:35.332875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:35.336100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 461/898] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:35.345996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.381882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.383223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.383945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.384367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.385779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:35.386466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.389909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.413762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 438/899] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:35.417822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.523700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2e006-165e-44f9-a9b6-230bb6073c9d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.541767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d081c51-8722-4f74-a9f8-cfcbf6562e56 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.555750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:35.563543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:35.579097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.579750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.580099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.580331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.581035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:35.581339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.582845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.589817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.590854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.591320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.591777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.593875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 462/900] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:35.600062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.613034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.616688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:35.635054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:35.636302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:35.637292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:35.643071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 439/901] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:35.644595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.747962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1d691-78e9-4ea9-b058-9c3c9e2f307c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.759534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9a41-04c9-4057-b1f8-4dc45a266cc2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:35.797693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.798313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.798589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.798858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.799485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:35.799778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.801153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.806324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:35.807211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:35.807699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:35.808197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:35.809152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:35.809687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:35.811399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:35.812629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 463/902] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:35.834670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 440/903] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:35.843359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:35.907219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:35.971275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d35c02-8d95-4130-9a8b-0458b8c9bf04 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:36.015890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:36.017009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.017587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.018267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.096490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:36.098038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:36.155235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:36.215390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 464/904] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:35 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:36.221442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:36.254697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:36.346744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04e52e-486f-48cd-87b6-986f8f072ce8 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:36.385924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:36.386962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.387447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.387922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.401349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:36.403873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-5fa9e994-e300-45eb-80c1-7d0907e2060d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:36.421062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:36.430271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:36.431226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:36.431226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:36.431226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:36.431226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 441/905] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:36.431464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:36.473876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a94f-cb96-40b9-8ef6-c51435fd2c44 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.473876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a94f-cb96-40b9-8ef6-c51435fd2c44 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.473876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643a94f-cb96-40b9-8ef6-c51435fd2c44 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.475373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 442/906] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:36.481000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.481404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.481917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.503146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:36.504596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:36.550863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95bd0c-6dee-4c45-bb39-71244069b34e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:36.589900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 443/907] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:36.593488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:36.594302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.594727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.595144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.595957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:36.596425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:36.597976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:36.613851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 465/908] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:36.620020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:36.635427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:36.763262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021b88f-8884-400f-9f7a-8100c03a745b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:36.800827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:36.801871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:36.803251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:36.803760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:36.811379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:36.813201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:36.826012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:36.826927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:36.827811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:36.830240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 444/909] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:36.839490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:36.855566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:36.896881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:36.964597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef37a6-e0be-40bd-bbea-5c114df96bcd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:36.973676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:37.006433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.007391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.007872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.008300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.009198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36866e7-9fdd-4d17-8e29-3bacd51e3e38 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.009338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:37.009852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.011540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.033125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 466/910] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:37.035567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.061123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.061809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.062086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.062355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.062944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.064334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:37.074186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:37.075214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 21:03:37.076108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.080422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 445/911] 10.222.0.99 () {56 vars in 1132 bytes} [Fri Sep 2 21:03:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:37.081419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.136424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:37.172254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bfade4-dcb7-4164-963b-746bed0fb96b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.197850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bedba-9550-43ed-b054-ff55d01bdfa1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.219364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.219997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.220271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.220552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.221247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] RBAC: Adding query filter params (role.id=afbe00c9b5694a6db91d8c79e51d73e2, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=b8c976a6a3bf48a6a518bdd7b604087a) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:37.221552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.223028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3177c-a3ae-4292-b440-b5d586b45f06 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.229746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 467/912] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:37 2022] GET /identity/v3/role_assignments?user.id=b8c976a6a3bf48a6a518bdd7b604087a&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=afbe00c9b5694a6db91d8c79e51d73e2 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:37.230799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76833025-ec05-48b3-84fc-7b6596d95abe None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.231080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76833025-ec05-48b3-84fc-7b6596d95abe None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.231352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76833025-ec05-48b3-84fc-7b6596d95abe None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.236869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 468/913] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:37.240191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.241108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.241498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.259170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.261443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.261955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.262765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.263646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:37.264109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.265698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.267202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:37.267547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:37.268888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:37.278017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 446/914] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:37.279786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.366323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 469/915] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:37.411879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a840d3-53be-40d1-a6de-c7c94498373c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.451134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.451747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.452059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.452332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.452967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:37.453263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.454681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.461999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 447/916] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:37.467926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.537743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:37.585364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668dd95-707d-4ae5-95f5-1543d7c1fcf0 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.628409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.628975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.629250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.629517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.630776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.632554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.680054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.762419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:37.764478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 470/917] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:37.814265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcada9-56ac-4cfa-b978-3d60364625b4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:37.866718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:37.867456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.867735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.868055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.868645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:37.870132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b10a4ea9-a194-44f2-9603-04ff8d436e7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:37.882789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:37.883844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 21:03:37.884684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:37.887717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 448/918] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:37.893515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:37.919751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b0c9-f00d-4fd1-9fc9-393ee1e0f4b5 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.920061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b0c9-f00d-4fd1-9fc9-393ee1e0f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.920285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8b0c9-f00d-4fd1-9fc9-393ee1e0f4b5 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.922916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 449/919] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:37.927953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:37.928248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:37.928592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:37.948227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:37.949418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:38.029845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 450/920] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:38.035213 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad09755-7313-4e6c-a04c-4b76b122e22c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:38.058927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:38.074524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:38.075514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:38.075977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:38.076427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:38.077271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:38.077851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:38.079398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:38.084975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 471/921] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:38.090514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:38.153502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:38.202079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299397e6-3e92-4b7d-8621-9234e243ca9c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:38.238640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:38.239565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:38.239878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:38.240739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:38.241380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:38.241670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:38.243163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:38.249295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 451/922] 10.222.0.99 () {56 vars in 1114 bytes} [Fri Sep 2 21:03:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:38.254621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:38.317117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:38.384048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399d9f6-9cbb-4231-9a0c-6eb69ea4d264 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:38.435777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:38.436713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:38.437143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:38.437906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:38.439044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:38.440566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23551e62-cbc8-4dd4-95a9-145c5b6f9d59 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:38.440716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05645a-e4b4-4540-87e5-ba7e0bc1a09a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:38.460212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:38.478966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:38.479655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:38.479998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:38.480353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:38.481073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:38.482527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b557108-c127-497e-822b-1e0baf6a8357 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:38.524876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 472/923] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:38.590644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 452/924] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:38 2022] POST /identity/v3/services => generated 241 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:03:39.315378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c4af8-f118-4cf2-806f-f4fd832e0369 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:39.315941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c4af8-f118-4cf2-806f-f4fd832e0369 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:39.315941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c4af8-f118-4cf2-806f-f4fd832e0369 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:39.317614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 473/925] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:39.324332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7434fe-cfac-48a9-a28f-7b6760299e47 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:39.324801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7434fe-cfac-48a9-a28f-7b6760299e47 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:39.324876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7434fe-cfac-48a9-a28f-7b6760299e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:39.354264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3b7434fe-cfac-48a9-a28f-7b6760299e47 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:39.355486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3b7434fe-cfac-48a9-a28f-7b6760299e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:39.442185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 453/926] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:39.724060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c4af8-f118-4cf2-806f-f4fd832e0369 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:39.775800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:39.855015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c4af8-f118-4cf2-806f-f4fd832e0369 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:39.856290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c6ef1-65a3-4b85-be99-afe7095ab997 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:39.856643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c6ef1-65a3-4b85-be99-afe7095ab997 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:39.856923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c6ef1-65a3-4b85-be99-afe7095ab997 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:39.858617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 454/927] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:39.864125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:39.864408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:39.864643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:39.884783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:39.886065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:39.897999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:39.898647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:39.898927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:39.899202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:39.910101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:39.916424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:39.917384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:39.918276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:39.922687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 474/928] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:03:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:39.923574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:39.972143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 455/929] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:40.033228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b909e-c6a4-462e-a763-7845c9f56038 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.038479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.071041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.071650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.071926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.072192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.072859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:40.073152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.074546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.084344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 475/930] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:03:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:40.090950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.138280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.214779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63422a8e-eca1-4d1b-b801-e47c340e271d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.254703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.256158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.256438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.256714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.266962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.272553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.274083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:40.294523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:40.295574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:40.296471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:40.296471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:40.296471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:40.296471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 456/931] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:40.303144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.404255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be3b9-86c3-4c71-ab5f-72ad16debf99 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.406148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.428585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369435-629b-4b1d-9bd4-65ac0b59fb81 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.449244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.450092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.450474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.450847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.451554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.453116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:40.464911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:40.472789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:40.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 476/932] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:40.476321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.483665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.484347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.484698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.484975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.485639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:40.485977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.487562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.505970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 457/933] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:40.514739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.527505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.624058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bdaf1-6ab8-4ad5-a863-082922bf9f6b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.627937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5271c7-3d0a-4271-b331-6e4b8b050704 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.664122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.665023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.665538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.665989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.666525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.666956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:40.667157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.667432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.667703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.667859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.669387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e437f0-60c3-41be-8832-a1d54e7ff2f7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.675305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 477/934] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:40.676745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26fa8f-b0b4-40ed-8c2a-c7db06e82b65 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.677342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26fa8f-b0b4-40ed-8c2a-c7db06e82b65 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.677814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26fa8f-b0b4-40ed-8c2a-c7db06e82b65 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.679200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 478/935] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:40.681024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.682629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.684402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.684987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.685416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:40.695719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:40.696959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:40.698158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:40.701467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 458/936] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:40.702582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.706349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.709758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:40.711211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:40.795995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 479/937] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:40.797089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:40.826308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379898f8-f0d8-4763-9205-2a4ec53f44b4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.830323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:40.865244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:40.866033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.866309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.866577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.867268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:40.867568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:40.868940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f58729f-5fcf-4637-aadf-549267ea792f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:40.874406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 459/938] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:40.875771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7107a6-4166-4e63-9663-6a82bb509347 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.876052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7107a6-4166-4e63-9663-6a82bb509347 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.876325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7107a6-4166-4e63-9663-6a82bb509347 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.877750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 460/939] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:40.884058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:40.884404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:40.884767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:40.912039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:40.914013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:40.950165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65bbd8-cedf-4b99-8cc8-3549f477b4fe None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:40.999172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.013743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 461/940] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:41.016121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.016761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.017033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.017299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.018276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:41.018578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.020089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.041936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 480/941] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:41.047973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.074919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.170350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a510-1c89-4f62-a597-0a1ee012aa8e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.176079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.210151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfcf55-19c3-4992-bf26-a556a25ecdeb None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.216473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.217421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.218051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.218544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.233678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.236138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.254140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.255524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.256085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.256611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.257389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:41.257541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:41.258458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:41.259328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:41.259328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:41.259328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:41.259504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.261663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 462/942] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:41.266267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING keystone.server.flask.common [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 02 21:03:41.267131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.322148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.323321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.sql.core [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 21:03:41.323321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 21:03:41.323321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [parameters: {'id': 'ca173880de1f49afbbaa5af19deeec41', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 21:03:41.323321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81842) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca173880de1f49afbbaa5af19deeec41.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca173880de1f49afbbaa5af19deeec41. Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:03:41.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result.read() Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 02 21:03:41.346574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application yield resource Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise self.value Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application session.commit() Sep 02 21:03:41.348340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.session.flush() Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application compat.raise_( Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 21:03:41.349267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 21:03:41.350745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result.read() Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:03:41.351539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [parameters: {'id': 'ca173880de1f49afbbaa5af19deeec41', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.354105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:41.355596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ca173880de1f49afbbaa5af19deeec41. Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 481/943] 10.222.0.99 () {60 vars in 1140 bytes} [Fri Sep 2 21:03:41 2022] POST /identity/v3/projects => generated 262 bytes in 273 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:41.357929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.379608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999cba-4341-40b9-acc9-ee5db98b8eae None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.416353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.417016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.417230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.417493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.418165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:41.418466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.419915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.434954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 463/944] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:41.436199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.476606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62148-62a8-4a4d-a687-1960e14922a7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.524240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.525145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.525571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.526514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.527294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.529412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.532294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:41.543260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:41.550810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.551655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 482/945] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:41.557322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.558858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3fa61-3395-40b5-98f5-30c948385cc5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.572016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.596867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.597559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.597889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.598130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.606780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.608311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:41.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:41.624670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:41.625549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:41.630464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 464/946] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:41.631368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.674847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0933-6250-4638-a077-2af0d8847421 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.717912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.718556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.718843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.719114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.727769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.729163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:41.740833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:41.741790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:41.742881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:41.746974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 483/947] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:41.748057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.751984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7e18c-eff7-42c6-906f-10e1ad98df9a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.792640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.793377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.793659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.793955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.794658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:41.795081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.796448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.802081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 465/948] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:41.807749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.813511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.843992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:41.860796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dee7d-cb8c-41b8-9cbc-cd09cf528324 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.906582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.907197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.907498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.907743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.908378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:41.908698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.910138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:41.920537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9de53-14d6-459f-9ca4-3391dd27c2fb None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:41.941757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 484/949] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:41.941757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:41.979104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:41.979637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:41.979912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:41.980182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:41.980944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:41.981181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:41.983534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:42.002386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 466/950] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:42.011769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:42.089807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc346a-8dff-4a09-a32f-7c6766606af9 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:42.119008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:42.140991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:42.141897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.142178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.142450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.143268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:42.143911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:42.145911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:42.149513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:42.156613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f37d-6fdb-4cbb-9e89-37afb833d0e5 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:42.159422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 485/951] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:42.160626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:42.201159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:42.201870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.202137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.202410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/b8c976a6a3bf48a6a518bdd7b604087a/roles/afbe00c9b5694a6db91d8c79e51d73e2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.204897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=b8c976a6a3bf48a6a518bdd7b604087a, role_id=afbe00c9b5694a6db91d8c79e51d73e2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:42.206417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:42.290045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262e55b-1e1e-41a7-b581-3411b87c432e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:42.335974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:42.337062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.337585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.338117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.339118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] RBAC: Adding query filter params (interface=public, service_id=d9d4f8c81dfa4eac860cc211562d4056, region_id=RegionOne) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:42.339742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:42.341594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0657d8-8837-4775-b4f1-3588f9d85180 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:42.341869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 467/952] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:42 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/b8c976a6a3bf48a6a518bdd7b604087a/roles/afbe00c9b5694a6db91d8c79e51d73e2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:03:42.350242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 486/953] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:42 2022] GET /identity/v3/endpoints?service_id=d9d4f8c81dfa4eac860cc211562d4056&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:42.361806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:42.500979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabbd8d-7857-4680-a651-81bb4391ec95 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:42.543258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:42.543795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.544079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.544486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.545097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:42.546519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9e495-bb62-4ddd-beb6-ed134ec537d5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:42.554447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 468/954] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:42 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:03:42.923851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a167-5b15-44ce-8a94-6a14cd6acb00 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.924369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a167-5b15-44ce-8a94-6a14cd6acb00 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.924369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a167-5b15-44ce-8a94-6a14cd6acb00 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.926028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 487/955] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:42.932599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bc89a-dbdc-461d-88dc-3447fc8b2776 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:42.932889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bc89a-dbdc-461d-88dc-3447fc8b2776 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:42.933166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bc89a-dbdc-461d-88dc-3447fc8b2776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:42.953283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e46bc89a-dbdc-461d-88dc-3447fc8b2776 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:42.954534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e46bc89a-dbdc-461d-88dc-3447fc8b2776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:42.970051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:43.033388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 469/956] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:43.324731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a167-5b15-44ce-8a94-6a14cd6acb00 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:43.384483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:43.443645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a167-5b15-44ce-8a94-6a14cd6acb00 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:43.484138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:43.484767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:43.485038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:43.485305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:43.486018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:43.487423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0085d03-9975-4c58-b31c-11d71dc3c89c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:43.574039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 488/957] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:43 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:44.377400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18d75a-d64e-43fb-a62c-371befcb65ac None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.378096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18d75a-d64e-43fb-a62c-371befcb65ac None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.378096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18d75a-d64e-43fb-a62c-371befcb65ac None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.379690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 470/958] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:44.386521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40b1f-263b-42d8-9fb1-c333bf0814b0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.387167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40b1f-263b-42d8-9fb1-c333bf0814b0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.387167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40b1f-263b-42d8-9fb1-c333bf0814b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.410533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ebb40b1f-263b-42d8-9fb1-c333bf0814b0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:44.414045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ebb40b1f-263b-42d8-9fb1-c333bf0814b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:44.486969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905cb4b-263a-46a8-ac2a-1d0a31d33ba7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.487339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905cb4b-263a-46a8-ac2a-1d0a31d33ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.487529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905cb4b-263a-46a8-ac2a-1d0a31d33ba7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.489202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 471/959] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:44.494680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f8a00-191c-4147-ab59-df3aba6807a7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.495003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f8a00-191c-4147-ab59-df3aba6807a7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.495215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f8a00-191c-4147-ab59-df3aba6807a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.514438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-861f8a00-191c-4147-ab59-df3aba6807a7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:44.515634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-861f8a00-191c-4147-ab59-df3aba6807a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:44.566237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 489/960] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:44.597898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 472/961] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:44.707722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fc34-4fcb-43f8-9305-1d321407dd8b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.708503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fc34-4fcb-43f8-9305-1d321407dd8b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.708846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fc34-4fcb-43f8-9305-1d321407dd8b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.710598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 490/962] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:44.717438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:44.717438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:44.717438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:44.737582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:44.739760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:44.823741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 473/963] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:44.893429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fc34-4fcb-43f8-9305-1d321407dd8b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:44.932583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:44.932803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:44.943983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.009469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fc34-4fcb-43f8-9305-1d321407dd8b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.038061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c2d49-cc13-44b5-aafd-11f8b3a388ba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.047199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.047820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.048160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.048435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.049143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.050627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.077205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.077872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.078147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.078414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.090452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:45.095508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:45.101476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 474/964] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:03:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:45.104187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.107786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.sql.core [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 21:03:45.107786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 21:03:45.107786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [parameters: {'user_id': 'cb9ebbbd9838402a9137ae446598e437', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 21:03:45.107786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81842) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result.read() Sep 02 21:03:45.116360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:03:45.117326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application yield resource Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application next(self.gen) Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise self.value Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application session.commit() Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 21:03:45.118696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.session.flush() Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application compat.raise_( Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 21:03:45.119664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 21:03:45.120836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application conn.query(q) Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application result.read() Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 21:03:45.122352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cb9ebbbd9838402a9137ae446598e437', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:45.124358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 02 21:03:45.125435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 21:03:45.126713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.133572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 491/965] 10.222.0.99 () {60 vars in 1131 bytes} [Fri Sep 2 21:03:44 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:45.133572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.178663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.219603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af550ba-7c08-4b61-8f73-56305eb348a6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.257768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.262688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d2e25-999b-4493-9e33-a1ba8d295a81 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.279095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.280416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.280416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.280416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.281106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:45.281410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.283058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.294228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 475/966] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:45.295095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.306054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.307052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.307478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.307890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.309163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.310935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:45.323043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:45.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.330785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 492/967] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:45.331636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.410153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f75648-b1d6-4ead-9c0e-0f524117402d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.440432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.447385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.448026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.448431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.448606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.450905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454ab9-f725-4460-b253-dba9b82609d7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.457393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.459461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:45.472219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:45.479413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 476/968] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:45.480378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.495144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.496044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.496315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.496583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.508920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.510432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:45.527498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:45.528805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:45.529706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:45.529706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:45.529706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:45.529706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 493/969] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:45.529706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.544007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.602979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585f446-1a38-4707-9590-b6bf71fb4b12 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.645135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.646477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.646758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.647027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.647661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:45.647989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.649379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.658878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 477/970] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:45.658878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.659143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a845e-2f2d-4954-8244-482ea70a92d1 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.693301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.694034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.694315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.694586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.695289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:45.695588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.697030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193f896-c3b2-4fb4-9dac-11787e51f641 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.719268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.737353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 494/971] 10.222.0.99 () {56 vars in 1101 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:45.738810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3da4b-1a9d-462e-a8a0-57ac79ec0164 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.745835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3da4b-1a9d-462e-a8a0-57ac79ec0164 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.745835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3da4b-1a9d-462e-a8a0-57ac79ec0164 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.745835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 495/972] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:45.745835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3da4b-1a9d-462e-a8a0-57ac79ec0164 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.770578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0feaa-b1ae-45ff-b066-3da515fd325c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.811846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.812731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.813006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.813278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.813970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:45.814270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.815682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52d135-22fe-46a3-bd76-7dccbe1ad68a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.830485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 478/973] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:45.831591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.831869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.832136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.842165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:45.852743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:45.865610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:45.886794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3da4b-1a9d-462e-a8a0-57ac79ec0164 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:45.924119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:45.924729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:45.925009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:45.925276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:45.925926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:45.926235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:45.928705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:45.934747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 479/974] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:45.935160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 496/975] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:45.938279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.943955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:45.999570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.072064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d594a5-d35f-434e-afb6-e8d4cbd99563 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.075438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1321702-bff7-4ea1-9ab6-4a57e73eddc0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.107830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.112698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.113159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.113376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.113648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.113851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.114011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.122582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.122582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.122582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.122582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.122914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.124260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:46.138830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:46.140074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:46.141586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 480/976] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:03:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:46.146852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.255152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af8080-f853-44e7-8d14-2706d7957660 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.257859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.278969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 497/977] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:03:46.279907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.315245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.315869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.316141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.316403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.318804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:46.318804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.320272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.346063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 481/978] 10.222.0.99 () {56 vars in 1149 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:46.347236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.416678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6abc1-90da-4f94-9d0a-325d2c0791fc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.465534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.466271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.466487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.466762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.467370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.468740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.476524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dea7be-8f7b-42e0-a365-bd7384d55a60 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:46.479909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:46.481618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:46.482798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 498/979] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:46.490286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.520739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.521347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.521623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.521927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.551255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.556921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.598871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.600390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.626670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b00344-b3d3-488e-b952-655f6e73b760 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.667864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.668551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.668747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.669144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.670118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:46.670482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.671892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.683300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 499/980] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:46.684415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.726636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 482/981] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:46.733111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.808086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.825633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d597c-371f-4b2b-a83d-4ca8d7b85588 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.845836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:46.863412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61bf80-5200-4f7b-9048-dd951b3a880a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:46.866261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.866949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.867295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.867653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.868359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:46.868726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.870468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.876114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 500/982] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:46.885403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:46.899598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:46.900233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:46.900789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:46.900955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:46.917831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:46.919300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:46.935554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:46.936810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:46.938093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:46.938093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:46.938093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:46.938093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 483/983] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:46.942859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:47.007629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f8d86-c1bb-4259-8431-622bb13fdae8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.048122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.048908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.049289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.049444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.050188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] RBAC: Adding query filter params (interface=public, service_id=f51cc54eb5a347cb8ba19fd1059ff90b, region_id=RegionOne) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:47.050482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.052856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.063468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 501/984] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/endpoints?service_id=f51cc54eb5a347cb8ba19fd1059ff90b&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:47.064455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:47.069023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f97b3-dd95-4eb6-b990-ada20c7b3ca0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.080747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:47.115042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.115988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.116446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.116919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.121036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:47.124102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:47.124192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.124288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.156765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 484/985] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:47.157931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:47.184768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ca81-dc50-47e9-9d27-3b1cc4505247 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.228405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.229050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.229329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.229782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.230304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.231717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.240777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 502/986] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:47 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:03:47.290543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0b54e-2cf5-4749-baa5-44a9cfd7a65d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.335474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.336376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.336753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.337130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.345500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.347285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:47.362573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:47.364034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:47.365248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:47.366447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 485/987] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:47.371182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:47.504200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf57c-5af9-4c8c-b1d9-2aa56895e9b6 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.516381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:47.545105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.545845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.546196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.546596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.547311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:47.547721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.549238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8166ae-06c7-4909-8acc-bdddd6ce2760 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.562292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 503/988] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:47.568866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:47.603345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:47.682536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58370b-9538-4b6a-b908-c839569e3b11 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:47.719730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:47.720399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:47.720674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:47.720937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:47.721658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] RBAC: Adding query filter params (role.id=afbe00c9b5694a6db91d8c79e51d73e2, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=b8c976a6a3bf48a6a518bdd7b604087a) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:47.721994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:47.723976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e5452-c30a-4609-9a79-5ebe6dcad73c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:47.737755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 486/989] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:47 2022] GET /identity/v3/role_assignments?user.id=b8c976a6a3bf48a6a518bdd7b604087a&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=afbe00c9b5694a6db91d8c79e51d73e2 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:48.343698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15856-2b2a-4d76-8f7d-9d292ba83d4a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:48.344116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15856-2b2a-4d76-8f7d-9d292ba83d4a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:48.344232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15856-2b2a-4d76-8f7d-9d292ba83d4a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:48.345794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 504/990] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:48.351680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:48.351985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:48.352233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:48.372426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:48.373837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:48.448160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 487/991] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:48.715140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15856-2b2a-4d76-8f7d-9d292ba83d4a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:48.827668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15856-2b2a-4d76-8f7d-9d292ba83d4a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:48.863095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:48.863656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:48.864455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:48.864878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:48.865698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:48.867116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:48.876893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:48.878018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:48.879073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 505/992] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:48.885395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:48.919703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:49.001602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b54dd-2a67-4ab2-bb96-1f0e09b8e96b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:49.037422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:49.038069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.038345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.038610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.039232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:49.039554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:49.040912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:49.045889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 488/993] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:49.052331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:49.085049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:49.158838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159acd06-31e9-4db5-b443-fb62591e8a6b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:49.194372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:49.194983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.195259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.195529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.196239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:49.196449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:49.197864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402ad0c-993a-4bf5-a37a-017e5587b5c7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:49.207851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 506/994] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:49.210134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:49.256450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ea7d-3ef3-4f41-b376-11cfbf70888b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.256787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ea7d-3ef3-4f41-b376-11cfbf70888b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.256997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0ea7d-3ef3-4f41-b376-11cfbf70888b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.258552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 507/995] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:49.263977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.264258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.264519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.284538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:49.286715 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:49.315266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:49.329217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48638de9-79de-4d78-b2f8-c77e8862fc72 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:49.367899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 508/996] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:49.372298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:49.372931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.373199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.373468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.374218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] RBAC: Adding query filter params (interface=admin, service_id=d9d4f8c81dfa4eac860cc211562d4056, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:49.374522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:49.375897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:49.381962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 489/997] 10.222.0.99 () {56 vars in 1249 bytes} [Fri Sep 2 21:03:49 2022] GET /identity/v3/endpoints?service_id=d9d4f8c81dfa4eac860cc211562d4056&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:49.387798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:49.423530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:49.503445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778d7e4-56ac-4c9a-a22e-e08d5b4b8ca4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:49.555021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:49.556082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.556565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.557045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.558208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:49.560812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35d44b-7c01-4c7c-b042-92e6135a898c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:49.581745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 509/998] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:49 2022] GET /identity/v3/services => generated 1727 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:03:49.657755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:49.749151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e61ce8-aa8c-4b00-beae-c8a7a72f75ab None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.749680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e61ce8-aa8c-4b00-beae-c8a7a72f75ab None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.749680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e61ce8-aa8c-4b00-beae-c8a7a72f75ab None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.751266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 510/999] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:49.758155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.758439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.758710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.778706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:49.779947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:49.806505 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3ecc-f690-4d89-bf80-1ce6dd9c4635 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:49.849502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:49.850178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:49.850524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:49.850710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:49.851295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:49.852677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:49.858694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 511/1000] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:49.862805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:49.863905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:49.864747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 490/1001] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:49.871296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:49.897394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:49.982472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c5c2a-073d-43f9-8fb9-a0aa65e7f06e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.022285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.022814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.023183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.023618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.023962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:50.024256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.025686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.034116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 512/1002] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:50.037561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:50.058048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:50.139218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:50.163046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3f5c8-0165-485b-a19f-cf3342265a94 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.200789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.201980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.202255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.202525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.203161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:50.203460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.205045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.216447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 491/1003] 10.222.0.99 () {56 vars in 1112 bytes} [Fri Sep 2 21:03:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:50.217578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:50.243090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:50.263768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efbac-5438-42d5-b979-adf170efcc53 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.305609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.306209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.306520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.307600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] PATH_INFO: `/v3/services/image` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.307872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.309188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.320087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:50.320409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:50.321391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:50.322287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 513/1004] 10.222.0.99 () {56 vars in 1105 bytes} [Fri Sep 2 21:03:50 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:50.327709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:50.344938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cb95d-36cb-40e9-90fd-d08608e7272b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.382958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.383564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.383838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.384105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.384767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.386184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6342c-9fb8-4857-a840-8ca31b5cde2b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.449913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f1bb-405e-4d88-969f-64d6e03acc6a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.484561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.485178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.485450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.485740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.486371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] RBAC: Adding query filter params (name=image) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:50.486664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.488042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.494352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 514/1005] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:50.500404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:50.517482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:50.554828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 492/1006] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:50.638720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657055d-c286-47f9-b1ee-b81d77d1265b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:50.685657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:50.686305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:50.686582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:50.686851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:50.687481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] RBAC: Adding query filter params (type=image) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:50.687776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:50.689157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2594c-a966-4238-8b0b-70f9c8d90fce None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:50.695270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 515/1007] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:51.592964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573693b-f7cd-4345-a357-396ca89f0be4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:51.593519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573693b-f7cd-4345-a357-396ca89f0be4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:51.593519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573693b-f7cd-4345-a357-396ca89f0be4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:51.594906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 493/1008] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:51.600740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:51.601174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:51.601274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:51.623756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:51.624941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:51.701771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 516/1009] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:51 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:51.970053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573693b-f7cd-4345-a357-396ca89f0be4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:52.085647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573693b-f7cd-4345-a357-396ca89f0be4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:52.122474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:52.123171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.123451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.123720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.124297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:52.125666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:52.128400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:52.135783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:52.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 21:03:52.137637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:52.139060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 494/1010] 10.222.0.99 () {56 vars in 1126 bytes} [Fri Sep 2 21:03:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:52.142963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:52.145516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:52.266328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef453d0b-3fe8-45af-ab29-fb2a4731c5f4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:52.310441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:52.311069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.311346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.311618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.312259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:52.312558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:52.313967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:52.325756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 517/1011] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:52.329261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:52.462983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf896-f22e-4216-9aa7-6aaad05caa09 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:52.500042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:52.500645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.500913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.501222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.501886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:52.502174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:52.503532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b52dd7-9cf6-4ca0-b4fb-14b500dd9546 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:52.514804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 495/1012] 10.222.0.99 () {56 vars in 1122 bytes} [Fri Sep 2 21:03:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:52.516275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:52.589489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:52.640119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83cc56-5eba-4485-a816-a1fc3df6d704 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:52.690824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:52.691436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.692373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.693265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.693989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:52.695369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16a354-7741-4bd4-9a12-9a017c4b210b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:52.699015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532265-6fbe-4159-accd-462e764e7264 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.699538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532265-6fbe-4159-accd-462e764e7264 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.699538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97532265-6fbe-4159-accd-462e764e7264 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.701306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 496/1013] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:52.702522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131fd6-942a-4678-9309-c5649542ee63 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.702846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131fd6-942a-4678-9309-c5649542ee63 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.703076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23131fd6-942a-4678-9309-c5649542ee63 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.704328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 497/1014] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:52.706306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.706582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.706857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.725402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:52.726626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:52.773178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 518/1015] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:52.774480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:52.774835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:52.775108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:52.795060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:52.796289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:52.805039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:52.828851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 498/1016] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:52.896771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 519/1017] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:53.124904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:53.178140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:53.239305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc727-3e51-4dc9-8588-08e375e4dad9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:53.252656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:53.286087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:53.286747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:53.287114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:53.287425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:53.300363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:53.306449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:53.307431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:53.308267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 499/1018] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:03:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:53.312739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96dc1-b756-45c9-9ffb-45020438cbc3 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:53.313675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:53.357649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:53.358325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:53.358607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:53.358904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:53.359546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:53.360931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:53.391065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:53.447269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287415a-1a82-4b2d-985a-5901ea913a28 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:53.484103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:53.484733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:53.485009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:53.485278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:53.486059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:53.486367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:53.487847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:53.501780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 500/1019] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:03:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:53.505066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 520/1020] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:53 2022] POST /identity/v3/services => generated 242 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:03:53.506682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:53.640614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3150f3-d38b-408f-8fc6-3c943b160e9a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:53.655905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:53.686329 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:53.686969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:53.687493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:53.687944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:53.700463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:53.701954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:53.717510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:53.719100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:53.720264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:53.720264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:53.720264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:53.720264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 501/1021] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:53.724539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:53.843069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e666dee-abf4-4782-8217-cf232ab51be7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:53.861556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:53.883513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:53.884130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:53.884406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:53.884677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:53.885326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:53.885626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:53.887027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:53.905025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 521/1022] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:53.906387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.022760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863603fe-070d-4a52-9790-79c13280ac5f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:54.060209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:54.060841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.061119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.061391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.068945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:54.070491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:54.082692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:54.084459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:54.085447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:54.086745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 502/1023] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:54.090249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.199173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823b13-adab-4aef-a1f7-3f3adbcfe7b3 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:54.236680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:54.237290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.237573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.237947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.238642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:54.239001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:54.240443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:54.264456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 522/1024] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:54.265751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.291239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:54.376586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2b8d-5cbd-4004-85dd-81d6b4b841ec None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:54.414291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:54.414909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.415183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.415450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.490821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:54.492280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:54.514663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:54.599678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 503/1025] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:03:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:03:54.606177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.647385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:54.717245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0579c-89f3-4cfc-929b-53dce15ceea0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:54.754620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:54.755873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.755873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.755873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.766548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:54.767824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1fa7b811-1a57-44b3-b388-274f16fd7a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:54.784077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:54.785194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:54.791613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:54.791613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:54.791613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:54.791613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 523/1026] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:54.791845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.849639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b4c2-0906-4d41-832f-0631310d3cde None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.850016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b4c2-0906-4d41-832f-0631310d3cde None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.850231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e8b4c2-0906-4d41-832f-0631310d3cde None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.851724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 524/1027] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:03:54.857025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.857571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.858081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.877846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:54.879842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:54.905754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f2fcf-0d8f-4f7c-9ebb-436453abb0d3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:54.927877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:54.947012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:54.947637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:54.947923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:54.948195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:54.948841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:54.949254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:54.950735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:54.966356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 504/1028] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:54.974683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:54.984170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 525/1029] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:55.088570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0b62c-be39-40df-82ea-ccbe1bd8093c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:55.132763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:55.133704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.134199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.134650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.142188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:55.143874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:55.156647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:55.157607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:55.158554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:55.164533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 505/1030] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:55.165313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:55.185194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:55.258213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:55.279675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc68c5-1e8b-441d-a378-d2f210580004 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:55.303423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:55.320486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:55.321111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.321441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.321678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.322501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:55.322699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:55.324099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:55.349847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 526/1031] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:55.351238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:55.385758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8896b2b4-50fa-4bf9-adcf-d43de6c2f7b7 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:55.423782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:55.424695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.424967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.425240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.426959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:55.428418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:55.430114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 506/1032] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:55.435920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:55.449267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:55.478154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1714ab-520b-4f5f-8914-27eca57a6a8f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:55.516982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:55.517650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.518012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.518277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.519004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] RBAC: Adding query filter params (role.id=afbe00c9b5694a6db91d8c79e51d73e2, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:55.519303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:55.521900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cd1b0-4af3-4b91-8f3c-2969c6ecd642 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:55.527623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 527/1033] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:03:55 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=afbe00c9b5694a6db91d8c79e51d73e2 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:55.528934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac253-a83e-43c8-97c2-1af458b7a0fc None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.529217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac253-a83e-43c8-97c2-1af458b7a0fc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.529487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ac253-a83e-43c8-97c2-1af458b7a0fc None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.531149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 528/1034] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:55.539521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.539938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.540471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.560226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:55.561639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:55.569113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c5cf-cee0-48ec-ac58-8074396ed597 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:55.580292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:55.617312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:55.617312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:55.617312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:55.617312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:55.617973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:55.618193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:55.651220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 529/1035] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:03:55.713737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 507/1036] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:03:55 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:03:55.924984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:56.004912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:56.041563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce291d-68d3-481a-be60-62309b5b2783 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:56.080535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:56.081169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:56.081450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:56.081742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:56.082944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:56.084305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:56.101970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:56.103998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:56.106489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:56.111671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 530/1037] 10.222.0.99 () {56 vars in 1105 bytes} [Fri Sep 2 21:03:55 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:56.112499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:56.230248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b3d6d-e4f0-4b8e-b2a4-615a520541aa None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:56.284962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:56.285819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:56.286302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:56.286772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:56.287915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] RBAC: Adding query filter params (name=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:56.288449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:56.290843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:56.300127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 508/1038] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:56.306364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:56.433537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab74-65b5-48d7-be4c-0aee214fcda5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:56.483932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:56.484528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:56.484842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:56.485065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:56.485684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] RBAC: Adding query filter params (type=image) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:56.486009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:56.487408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:56.489556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:56.494622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 531/1039] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:56.501455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:56.531964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:56.619479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53127-2450-4f00-ba6e-027585bab8af None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:56.656918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:56.657521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:56.657825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:56.658096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:56.658798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] RBAC: Adding query filter params (interface=public, service_id=cd7837a20c364d848ef5670361aa7b06, region_id=RegionOne) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:56.659096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:56.660512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5b6b-9067-4290-a706-cc92b7a8a1c3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:56.671287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 509/1040] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:03:56 2022] GET /identity/v3/endpoints?service_id=cd7837a20c364d848ef5670361aa7b06&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:56.672752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:56.782744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848540f-ba74-41e3-8ce0-89271c6b52ad None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:56.817504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:56.818141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:56.818414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:56.818679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:56.819272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:56.820621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc47f7-f0c2-43ff-96ea-56b3c8e6ef57 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:56.832761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 532/1041] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:03:56 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:03:57.476832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5dd7b-554d-4161-b051-de43a5dcef28 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:57.477475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5dd7b-554d-4161-b051-de43a5dcef28 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:57.477475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5dd7b-554d-4161-b051-de43a5dcef28 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:57.478875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 510/1042] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:57.484240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6aa52-b94b-45c8-bd34-5f4450308d62 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:57.484668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6aa52-b94b-45c8-bd34-5f4450308d62 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:57.484843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6aa52-b94b-45c8-bd34-5f4450308d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:57.503630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fdd6aa52-b94b-45c8-bd34-5f4450308d62 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:57.505647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fdd6aa52-b94b-45c8-bd34-5f4450308d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:57.565452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:57.577604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 533/1043] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:57.751139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a1782-f122-472a-84ad-0dfcb69e8135 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:57.751335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a1782-f122-472a-84ad-0dfcb69e8135 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:57.751592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a1782-f122-472a-84ad-0dfcb69e8135 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:57.752959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 511/1044] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:57.759100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:57.759385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:57.759660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:57.779556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:57.780747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:57.849449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a1782-f122-472a-84ad-0dfcb69e8135 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:57.861869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 534/1045] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:57.951666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:57.966262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a1782-f122-472a-84ad-0dfcb69e8135 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.001382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.001962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.002236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.002500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.013910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:03:58.018699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:58.021150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:03:58.022062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:58.025755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 512/1046] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:03:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:58.026619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.131967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026cf44-bda2-4217-831b-ed16be790417 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.138552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.156944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:58.169897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.170511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.170783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.171901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.171901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:58.172085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.173312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.182317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 535/1047] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:58.188061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.260042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94f35-fa87-4121-850c-41789fb5283a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.303954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.304599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.304874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.305139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.306296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.307727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.308914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5ab05-3072-4748-994f-a4ba25b79a17 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.309313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 513/1048] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:58.315076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.336604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:58.361443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.362120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.362400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.363052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.376187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.377650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:58.394751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:58.396161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:58.397091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:58.397091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:58.397091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:58.397091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 536/1049] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:58.401868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.442303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afe0cf-b8ea-44d3-9d2d-e0d5f193fd2c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.448042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:58.483960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.484704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.484704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.485157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.485928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.487416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.522197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272836-8c42-4dc0-ae98-9eae76e0337f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.559064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.562077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.562077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.562077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.562077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:58.562077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.562690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.574763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 537/1050] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:58.580518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.601751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 514/1051] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:03:58 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:03:58.703351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:58.712141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ba79e-ab0b-4a4d-98a5-c5f16ce0b7f0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.753201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.753997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.754899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.755179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.763578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.765044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-49817779-294f-48ea-9fad-ebc71ebcd482 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:03:58.778480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:58.779638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:03:58.780708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:58.784509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 538/1052] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:58.786116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:58.793842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e84ae-d305-47ae-81f3-fe013a180128 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.794212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e84ae-d305-47ae-81f3-fe013a180128 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.794374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e84ae-d305-47ae-81f3-fe013a180128 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.795910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 539/1053] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:03:58.801935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.802213 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.802487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.824910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:03:58.826251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:03:58.882008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:58.904047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 540/1054] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:03:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:03:58.915793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53bf74c-a701-4f95-aa10-80e0baaf7868 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:58.953497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:58.954149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:58.954432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:58.954698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:58.955429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:58.955729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:58.957779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:58.981761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 515/1055] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:58.984103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.081782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.098505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c518e-6b99-4ab7-b199-13f1c32ea4ae None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.138314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.138980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.139252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.139523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.158170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.160887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.198423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:03:59.200155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:59.201120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:03:59.202033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:03:59.202033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:03:59.202033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:59.202033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 541/1056] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:59.207704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.326264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd8b09-20be-4c7a-8ab5-62b212dc21c3 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.330098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08728d1-b6a6-4655-92f6-90b6b58cc3cc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.365836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.366469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.366611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.367336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.367621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.368286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:59.368592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.370440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.373830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.374714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.375143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.375581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] PATH_INFO: `/v3/services/image` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.376325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.377968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.381058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.387551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 542/1057] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:59.389222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:59.390249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 21:03:59.391117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:03:59.396131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 516/1058] 10.222.0.99 () {56 vars in 1105 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:03:59.397324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.397503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.507077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977133d-a261-4f44-9b29-77a62153c53f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.512357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41ead-6bde-445d-9f14-48fca3a55edc None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.546577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.547194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.547473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.547743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.548451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] RBAC: Adding query filter params (name=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:59.548732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.550220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.550533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.551340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.551708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.551992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.556534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 517/1059] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:03:59.559399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.560888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.562673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:03:59.576016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:03:59.577040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:03:59.578409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:03:59.582680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 543/1060] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:03:59.585597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.634939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.662130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.679515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3136-81db-4b6a-a975-7129e4733a10 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.707471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fae67-dd22-4d8b-9b73-6b2bb2a77407 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.725583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.726243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.726520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.726791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.727427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] RBAC: Adding query filter params (type=image) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:59.727722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.729097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.746788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 518/1061] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:59.748237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.752756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.753367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.754203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.754646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.756727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:03:59.757036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.758647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.774002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 544/1062] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:03:59.779951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:03:59.866245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6d2e-61c0-43f2-9a49-4a9c7521a808 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.892600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.892938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58874e95-4b11-491c-82a6-7ffe41a0aa4e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:03:59.905993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.908518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.908518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.908518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.908518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.910047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0dd383-abb0-4d38-8d85-b9131508bad0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:03:59.913246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:03:59.931339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:03:59.932079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:03:59.932357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:03:59.951983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/afbe00c9b5694a6db91d8c79e51d73e2` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:03:59.951983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=2d95be82e9e04470902aef0d9bab1919, role_id=afbe00c9b5694a6db91d8c79e51d73e2)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:03:59.951983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b37398-eaa5-4d22-b9b1-74686b53d6a1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:00.071750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 519/1063] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:03:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:04:00.091574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 545/1064] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:03:59 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/2d95be82e9e04470902aef0d9bab1919/roles/afbe00c9b5694a6db91d8c79e51d73e2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:04:00.718528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d69fb-13ac-47c3-9d09-cd496a829847 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:00.719055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d69fb-13ac-47c3-9d09-cd496a829847 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:00.719055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d69fb-13ac-47c3-9d09-cd496a829847 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:00.720348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 520/1065] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:00.725977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692189-34eb-4466-8805-6622b616b465 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:00.726476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692189-34eb-4466-8805-6622b616b465 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:00.726563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd692189-34eb-4466-8805-6622b616b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:00.748290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fd692189-34eb-4466-8805-6622b616b465 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:00.749481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fd692189-34eb-4466-8805-6622b616b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:00.830278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 546/1066] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:01.115761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d69fb-13ac-47c3-9d09-cd496a829847 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:01.233883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d69fb-13ac-47c3-9d09-cd496a829847 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:01.285888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:01.286534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:01.286813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:01.287083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:01.302583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:01.307471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:01.308500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:01.309486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 521/1067] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:01.315108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd692189-34eb-4466-8805-6622b616b465 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:01.464588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd692189-34eb-4466-8805-6622b616b465 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:01.468724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:01.527094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:01.528109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:01.528587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:01.529059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:01.530325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:01.530790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:01.533282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:01.553085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 547/1068] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:01.555294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:01.601531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:01.677873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc3ec5-11b7-4d1c-a8f2-fa4b9a1e916b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:01.716059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:01.716606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:01.716877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:01.717144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] PATH_INFO: `/v3/users/111c95fa60274399ba91630f0274996e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:01.732490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] RBAC: Authorizing `identity:get_user(user_id=111c95fa60274399ba91630f0274996e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:01.733971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:01.740406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 522/1069] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:01 2022] GET /identity/v3/users/111c95fa60274399ba91630f0274996e => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:01.742070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:01.859206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b439f6-0623-4685-be40-da6f7775119e None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:01.895444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:01.896110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:01.896405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:01.896653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:01.935344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:01.974337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:01.975709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef2206-70d0-44dc-b48e-7e35ce528e9b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:02.084918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 548/1070] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:02.090890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.095036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d052e-5af3-4230-9a76-0e6e044f75a5 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.095320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d052e-5af3-4230-9a76-0e6e044f75a5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.095621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d052e-5af3-4230-9a76-0e6e044f75a5 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.097022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 549/1071] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:04:02.102862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f94a-354f-40c9-bfb9-a721e6d2a6e4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.103142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f94a-354f-40c9-bfb9-a721e6d2a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.103416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f94a-354f-40c9-bfb9-a721e6d2a6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.121781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f471f94a-354f-40c9-bfb9-a721e6d2a6e4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:02.123000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f471f94a-354f-40c9-bfb9-a721e6d2a6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:02.187707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.194889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.195640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 550/1072] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:04:02.197879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26760748-c939-429f-a390-e8faffb2b92f None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.197879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26760748-c939-429f-a390-e8faffb2b92f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.197879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26760748-c939-429f-a390-e8faffb2b92f None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.202537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 551/1073] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:02.203323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4b5f6-5b9d-4f5d-8a2c-61c38201c73d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:02.204276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.204559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.204834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.224751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:02.226748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:02.246597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:02.247250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.247525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.247790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.249121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:02.250764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:02.252789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 523/1074] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:02.259693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.311496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 552/1075] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:04:02.375850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf2c58-834a-4b4c-91a6-dae3444eb268 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:02.412185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:02.412789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.413068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.413341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.414092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=81d0fd935bb242f5946e250b344e4b3a, user.id=111c95fa60274399ba91630f0274996e) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:02.414401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:02.415835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:02.421998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 524/1076] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/role_assignments?user.id=111c95fa60274399ba91630f0274996e&scope.project.id=81d0fd935bb242f5946e250b344e4b3a&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:02.484434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.584928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.615387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e5bc8-7fb2-4274-81c2-70c2ff06a4ec None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:02.620789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.662082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.673045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:02.673748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.674296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.674577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.687228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:02.693352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:02.694544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:02.695403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 553/1077] 10.222.0.99 () {56 vars in 1120 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:02.700672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.719222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f800e-3d7d-4ad9-a5f5-ec3130c19066 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:02.764745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:02.765829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.766115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.766678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.819477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da2866-28fd-4777-a2e9-b99f2d0c037a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:02.857558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:02.858100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:02.858225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:02.858464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:02.858734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:02.859400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:02.859701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:02.860526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:02.861052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:02.870777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 554/1078] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:04:02.876593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.896829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.930735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:02.977757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 525/1079] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 21:04:02.983229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:02.999850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a886b-fee0-46f2-84ed-ef9179ffa252 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.047104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.047948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.048365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.048773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.064530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.066869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:04:03.086485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:03.088032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:04:03.089087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:04:03.089087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:03.089087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:03.089087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 555/1080] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:03.094428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:03.112734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80253ea9-15c9-458f-b591-bce0a7f7956e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.142680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:03.151383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.152042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.157810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.157810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.164004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.165468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:04:03.181028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:03.182087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:04:03.183080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:04:03.183080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:03.183080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:03.183868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 526/1081] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:03.188414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:03.207821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:03.211299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910128-5aa2-4328-8432-847de5d6567f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.253117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.254935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.255415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.255593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.262422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:03.262422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.262422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.274655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 556/1082] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:04:03.280634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:03.308128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df22e-3c85-43cf-9097-a6df8b838fe3 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.349272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.350083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.350528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.350892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.351728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:03.352019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.353477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.377742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 527/1083] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:04:03.407881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b21b-3fc3-4300-aabd-626084eb6e5d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.452193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.452853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.453134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.453407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.462965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.464435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:04:03.477879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:03.478931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 21:04:03.479882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:03.482089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 557/1084] 10.222.0.99 () {56 vars in 1093 bytes} [Fri Sep 2 21:04:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:03.485123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:03.537148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:03.602530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd499a-ce2d-4abf-9aab-f1984e1ee56d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.641677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.642313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.642590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.642867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.643596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:03.643896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.645366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.664723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 528/1085] 10.222.0.99 () {56 vars in 1133 bytes} [Fri Sep 2 21:04:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:04:03.670971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:03.696776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:03.779516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3aef41-25ee-4e2c-9abd-f6950c0d57e1 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:03.814675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:03.815281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:03.815552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:03.815822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:03.889448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:03.890880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:03.915188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:04.005686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 558/1086] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:04.006925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.105751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:04.109389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a107f-a70e-43e1-aaa1-8bd44ddd7455 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.146138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.146673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.146943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.147204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.157891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:04.159402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:04:04.174147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:04.175206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:04:04.180958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:04:04.180958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:04:04.180958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:04.180958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 529/1087] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:04.181201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.288573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9228e-6a84-48f4-9795-cfc26ae484c7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.308836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:04.324398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.325007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.325321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.325590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.326256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:04.326555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:04.327912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff67263-d115-4bb5-ba90-56a5230595b7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:04.340585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 559/1088] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:04.346313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.417830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26e1f-26dd-4b65-8fbb-4b8c1d5be89d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.418068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26e1f-26dd-4b65-8fbb-4b8c1d5be89d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.418348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c26e1f-26dd-4b65-8fbb-4b8c1d5be89d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.419815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 560/1089] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:04:04.424896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.425200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.425555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.444661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:04.445991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:04.455365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb64faa-214d-44d8-9b83-07d2798f6654 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.490267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.490896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.491165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.491433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.498426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:04.499852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:04.516449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 561/1090] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:04:04.516708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:04.517791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:04:04.518820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:04.523092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 530/1091] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:04.524017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.627836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453dd3d-39dc-49e2-8da6-ca3d71e867c8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.648091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:04.673965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.674436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.674711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.674976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.676598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:04.676894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:04.678367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:04.690743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 562/1092] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:04:04.697285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.711625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:04.792514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:04.802504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa805a-0b0b-4147-8d19-6d4526b3c186 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.837352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.837996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.838269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.838538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.839273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] RBAC: Adding query filter params (role.id=afbe00c9b5694a6db91d8c79e51d73e2, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=2d95be82e9e04470902aef0d9bab1919) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:04.839591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:04.841025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:04.848094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 531/1093] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/role_assignments?user.id=2d95be82e9e04470902aef0d9bab1919&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=afbe00c9b5694a6db91d8c79e51d73e2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:04.907457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2892173-b07e-44c6-aa67-63d19532c05d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:04.946482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:04.947053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:04.947329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:04.947602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:04.961524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:04.966567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:04.967719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:04.968555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 563/1094] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:04.974806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:05.067797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:05.101547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0211-4da1-449e-8a07-474bb0505f3c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:05.154754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:05.155386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:05.155667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:05.155942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:05.156637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:05.156944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:05.158412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:05.176172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 532/1095] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:05.177602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:05.213952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:05.292523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f88a-2876-4162-9bec-59c4076c41ec None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:05.329640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:05.330293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:05.330567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:05.330836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] PATH_INFO: `/v3/users/111c95fa60274399ba91630f0274996e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:05.332080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] RBAC: Authorizing `identity:get_user(user_id=111c95fa60274399ba91630f0274996e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:05.333403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:05.335028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 564/1096] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:05 2022] GET /identity/v3/users/111c95fa60274399ba91630f0274996e => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:05.340933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:05.451817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc111e0-a134-4614-8068-982144d72970 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:05.490397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:05.490955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:05.491230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:05.491499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:05.492661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:05.494145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368bb3f-b770-4aa2-a0a7-8599ddc38bd8 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:05.498168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 533/1097] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:05 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:05.502739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:05.608815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:05.613963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff0b0-4822-47fa-97cf-5b82e1a86c73 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:05.650753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:05.651414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:05.651690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:05.651964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a/users/111c95fa60274399ba91630f0274996e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:05.653977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] RBAC: Authorizing `identity:create_grant(project_id=81d0fd935bb242f5946e250b344e4b3a, user_id=111c95fa60274399ba91630f0274996e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:05.655513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c984-d3a5-461f-85a0-42b2d8f7de83 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:05.728033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 565/1098] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:04:05 2022] PUT /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a/users/111c95fa60274399ba91630f0274996e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:04:07.726252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ecae3-5e11-4a4d-90bf-4137678363d1 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:07.726813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ecae3-5e11-4a4d-90bf-4137678363d1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:07.726813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ecae3-5e11-4a4d-90bf-4137678363d1 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:07.728093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 534/1099] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:07.733677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:07.734152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:07.734266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:07.753920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:07.755218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:07.824980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 566/1100] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:08.101680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ecae3-5e11-4a4d-90bf-4137678363d1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:08.105703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:08.203877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ecae3-5e11-4a4d-90bf-4137678363d1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:08.237812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:08.238563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:08.238831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:08.239091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:08.252118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:08.257099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:08.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:08.259178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 535/1101] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:08.264355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:08.353292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:08.365039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8704a-ad55-4707-aa34-7f3b8e5e4f3f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:08.399871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:08.400470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:08.400786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:08.401022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:08.401678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:08.402000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:08.403351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:08.412048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 567/1102] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:08.418007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:08.578308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:08.717205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ea4bb-6da1-4198-84c7-5fa9b377c884 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:08.749623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:08.750292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:08.750535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:08.750801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] PATH_INFO: `/v3/users/111c95fa60274399ba91630f0274996e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:08.751901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] RBAC: Authorizing `identity:get_user(user_id=111c95fa60274399ba91630f0274996e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:08.753908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:08.755508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 536/1103] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:08 2022] GET /identity/v3/users/111c95fa60274399ba91630f0274996e => generated 285 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:08.761738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:08.866559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb7c38-78ea-43ad-bfb1-47e1d64e8ed2 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:08.902487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:08.903102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:08.903376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:08.903647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:08.923685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:08.974707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:08.976158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:09.077794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 568/1104] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:09.083936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:09.210991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43654606-1158-4eb1-84ab-900319e2899d None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:09.238897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:09.243831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:09.247804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:09.247804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:09.247804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:09.247804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:09.247804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ec788-aed1-41fd-9efd-be7394bd130f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:09.249428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 537/1105] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:09 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:09.256278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:09.306790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:09.357220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76073041-1c25-4a9b-ae0a-abe60c439bc4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:09.392010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:09.392611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:09.392888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:09.393156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:09.394021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=81d0fd935bb242f5946e250b344e4b3a, user.id=111c95fa60274399ba91630f0274996e) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:09.394567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:09.395677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbdb94-b2c3-485b-8983-ba59e83b1266 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:09.401457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 569/1106] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:09 2022] GET /identity/v3/role_assignments?user.id=111c95fa60274399ba91630f0274996e&scope.project.id=81d0fd935bb242f5946e250b344e4b3a&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:11.382111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c968cf-de24-4559-af4a-dd1dd10a23de None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:11.382678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c968cf-de24-4559-af4a-dd1dd10a23de None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:11.382678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c968cf-de24-4559-af4a-dd1dd10a23de None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:11.383988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 538/1107] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:11.389331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:11.389769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:11.395365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:11.408724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:11.410026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:11.479843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 570/1108] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:11.752478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c968cf-de24-4559-af4a-dd1dd10a23de None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:11.855459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c968cf-de24-4559-af4a-dd1dd10a23de None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:11.889614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:11.890260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:11.890538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:11.890806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:11.891914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:11.893335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0696601-17c0-42a8-a073-fd569010a7be None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:11.895320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 539/1109] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:11.901223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:11.947524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:12.003430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93e814-bd75-4dc4-920d-d7aeaa92f8e9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:12.037064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:12.037668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:12.037979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:12.038252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:12.038889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:12.040308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236963f3-d19b-4a49-a66d-bb356d9c19e5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:12.150164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 571/1110] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:04:11 2022] POST /identity/v3/users => generated 286 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:04:14.121065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ae4c-76f9-49d2-9183-26099d0d1737 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:14.121733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ae4c-76f9-49d2-9183-26099d0d1737 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:14.121733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ae4c-76f9-49d2-9183-26099d0d1737 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:14.122974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 540/1111] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:14.128725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:14.129276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:14.129276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:14.148218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:14.149394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:14.216669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 572/1112] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:14.491053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ae4c-76f9-49d2-9183-26099d0d1737 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:14.558789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:14.593625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ae4c-76f9-49d2-9183-26099d0d1737 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:14.632393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:14.633015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:14.633287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:14.633553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] PATH_INFO: `/v3/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:14.634947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] RBAC: Authorizing `identity:get_role(role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:14.636298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:14.637599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 541/1113] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:14 2022] GET /identity/v3/roles/44671b36ffe9406e9372616fdd4b9400 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:14.643803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:14.724537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:14.793212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291107b-360d-4d85-8a67-992b8ae16ad2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:14.827818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:14.828442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:14.828714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:14.828980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] PATH_INFO: `/v3/users/3f7de05334754d75b56c8e9c0fd459a1` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:14.849313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3f7de05334754d75b56c8e9c0fd459a1)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:14.850786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:14.852487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 573/1114] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:14 2022] GET /identity/v3/users/3f7de05334754d75b56c8e9c0fd459a1 => generated 286 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:14.858410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:14.978500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f8a92-6fbc-4198-b302-9cf2a9153d69 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:15.012791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:15.013377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:15.013646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:15.013941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:15.017285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:15.079943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:15.081905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:15.178979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 542/1115] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:15.185076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:15.287148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:15.300739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f397-3042-4a8c-a4cf-5a1b859382a9 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:15.337227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:15.337801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:15.338083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:15.338353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:15.339460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:15.340878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77844a1-a049-492c-8b5d-ac08c0628c8a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:15.342708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 574/1116] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:15 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:15.349662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:15.385166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:15.451777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc70d9-f9ce-4cdb-99d2-f91513af20ee None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:15.486798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:15.487955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:15.488233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:15.488499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:15.489219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=81d0fd935bb242f5946e250b344e4b3a, user.id=3f7de05334754d75b56c8e9c0fd459a1) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:15.489519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:15.490980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11408b10-8a5e-464c-8837-f4e2b3f276e6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:15.496549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 543/1117] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:15 2022] GET /identity/v3/role_assignments?user.id=3f7de05334754d75b56c8e9c0fd459a1&scope.project.id=81d0fd935bb242f5946e250b344e4b3a&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:17.423600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f41b-e031-4024-b492-cb53f913834e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:17.424211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f41b-e031-4024-b492-cb53f913834e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:17.424211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540f41b-e031-4024-b492-cb53f913834e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:17.426138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 575/1118] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:17.431771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:17.432212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:17.432354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:17.450925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:17.452105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:17.517583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 544/1119] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:17.793777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540f41b-e031-4024-b492-cb53f913834e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:17.898158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540f41b-e031-4024-b492-cb53f913834e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:17.932097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:17.932724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:17.933002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:17.933272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] PATH_INFO: `/v3/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:17.934822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] RBAC: Authorizing `identity:get_role(role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:17.936249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:17.937573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 576/1120] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:17 2022] GET /identity/v3/roles/44671b36ffe9406e9372616fdd4b9400 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:17.944075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:17.974321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:18.044677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367ac66-328d-47da-b028-b55f84fcea71 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:18.078520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:18.079137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:18.079410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:18.079673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] PATH_INFO: `/v3/users/3f7de05334754d75b56c8e9c0fd459a1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:18.080887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] RBAC: Authorizing `identity:get_user(user_id=3f7de05334754d75b56c8e9c0fd459a1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:18.082336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:18.083892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 545/1121] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:17 2022] GET /identity/v3/users/3f7de05334754d75b56c8e9c0fd459a1 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:18.089809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:18.115460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:18.191346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a996d4-682e-4a97-a949-c60bad82c40f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:18.224511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:18.225128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:18.225399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:18.225664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:18.226821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:18.228781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eec662-77ab-42f0-bc0c-9105d535068c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:18.230437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 577/1122] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:18 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:18.236388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:18.338260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:18.342633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b80c4c-5879-4939-8a80-58362aa790b1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:18.381503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:18.382188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:18.382467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:18.382739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a/users/3f7de05334754d75b56c8e9c0fd459a1/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:18.384883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] RBAC: Authorizing `identity:create_grant(project_id=81d0fd935bb242f5946e250b344e4b3a, user_id=3f7de05334754d75b56c8e9c0fd459a1, role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:18.386356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8824cc-3511-4ba6-b077-2e844b5d5751 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:18.475810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 546/1123] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:04:18 2022] PUT /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a/users/3f7de05334754d75b56c8e9c0fd459a1/roles/44671b36ffe9406e9372616fdd4b9400 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:04:20.391127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d90d-ec58-4be8-b4c6-c1ceea77c2ca None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:20.391663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d90d-ec58-4be8-b4c6-c1ceea77c2ca None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:20.391663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d90d-ec58-4be8-b4c6-c1ceea77c2ca None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:20.393394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 578/1124] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:20.398605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:20.399062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:20.399257 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:20.418259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:20.424785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:20.484699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 547/1125] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:20.753802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27d90d-ec58-4be8-b4c6-c1ceea77c2ca None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:20.847765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:20.857420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27d90d-ec58-4be8-b4c6-c1ceea77c2ca None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:20.891301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:20.891940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:20.892419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:20.893672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] PATH_INFO: `/v3/roles/44671b36ffe9406e9372616fdd4b9400` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:20.895100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] RBAC: Authorizing `identity:get_role(role_id=44671b36ffe9406e9372616fdd4b9400)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:20.896466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:20.897817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 579/1126] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:20 2022] GET /identity/v3/roles/44671b36ffe9406e9372616fdd4b9400 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:20.903593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:20.988403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:21.000034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365096-1a8a-4605-a442-685ebdec62f1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:21.034981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:21.035567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:21.035838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:21.036102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] PATH_INFO: `/v3/users/3f7de05334754d75b56c8e9c0fd459a1` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:21.037207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] RBAC: Authorizing `identity:get_user(user_id=3f7de05334754d75b56c8e9c0fd459a1)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:21.038656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:21.040206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 548/1127] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:20 2022] GET /identity/v3/users/3f7de05334754d75b56c8e9c0fd459a1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:21.046075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:21.147058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab201e4-cbe9-4dcf-b8d2-6cec663df04c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:21.182575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:21.183187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:21.183463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:21.183729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:21.215136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:21.260850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:21.262399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:21.367271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 580/1128] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:21.373350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:21.473597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069967-e5e8-4ae9-a078-364c66dc1296 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:21.507682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:21.508296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:21.508568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:21.508837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] PATH_INFO: `/v3/projects/81d0fd935bb242f5946e250b344e4b3a` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:21.510123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] RBAC: Authorizing `identity:get_project(project_id=81d0fd935bb242f5946e250b344e4b3a)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:21.511782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e417ec-75bb-49f0-99ce-75f2911ead25 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:21.513464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 549/1129] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:21 2022] GET /identity/v3/projects/81d0fd935bb242f5946e250b344e4b3a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:21.520512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:21.578360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:21.618837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4811f62-d3e1-4e36-bb91-f106e376258b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:21.654732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:21.655337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:21.655616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:21.655889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:21.656604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] RBAC: Adding query filter params (role.id=44671b36ffe9406e9372616fdd4b9400, scope.project.id=81d0fd935bb242f5946e250b344e4b3a, user.id=3f7de05334754d75b56c8e9c0fd459a1) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:21.656913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:21.658377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4254e47-a103-41f3-a039-72ae980482d6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:21.664175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 581/1130] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:21 2022] GET /identity/v3/role_assignments?user.id=3f7de05334754d75b56c8e9c0fd459a1&scope.project.id=81d0fd935bb242f5946e250b344e4b3a&role.id=44671b36ffe9406e9372616fdd4b9400 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:23.615116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6bde-3f71-4e7a-86ca-076a4fb54990 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:23.615593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6bde-3f71-4e7a-86ca-076a4fb54990 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:23.615685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f6bde-3f71-4e7a-86ca-076a4fb54990 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:23.616974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 550/1131] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:23.622646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:23.623052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:23.623185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:23.641834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:23.643630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:23.711800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 582/1132] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:23.996258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f6bde-3f71-4e7a-86ca-076a4fb54990 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:24.022541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:24.109729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f6bde-3f71-4e7a-86ca-076a4fb54990 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:24.145448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:24.148635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:24.148913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:24.149183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:24.150862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:24.152346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4ff04-92a1-41ae-81a6-e2cfe1efc4b0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:24.154287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 551/1133] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:24.161554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:24.237053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:24.285544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c6a06-4729-43fc-8793-932439ff26fc None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:24.322318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:24.322925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:24.323196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:24.323495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:24.324150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:24.325650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d248f8-8e3f-4e89-95e9-9b2c354374f5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:24.443643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 583/1134] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:04:24 2022] POST /identity/v3/projects => generated 311 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:04:26.530198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0ad-1dc6-46d7-8601-ba0cb6451bda None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:26.530778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0ad-1dc6-46d7-8601-ba0cb6451bda None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:26.530778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0ad-1dc6-46d7-8601-ba0cb6451bda None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:26.532047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 552/1135] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:26.537695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:26.538174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:26.538370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:26.560797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:26.562048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:26.631745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 584/1136] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:26.917455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b0ad-1dc6-46d7-8601-ba0cb6451bda None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:27.007878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:27.016860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b0ad-1dc6-46d7-8601-ba0cb6451bda None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:27.050693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:27.051372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:27.051651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:27.051925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:27.053283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:27.055080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da5d71-89f4-4799-aa1e-ef22fa60a624 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:27.056639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 553/1137] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:04:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:27.063031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:27.099009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:27.165739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5504-3130-4732-9bc4-cf2b194be808 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:27.200568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:27.200783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:27.200919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:27.201185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:27.201862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:27.203316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58fb84-d999-44b3-a336-372322614423 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:27.377858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 585/1138] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:04:27 2022] POST /identity/v3/users => generated 286 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:04:29.370367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f641f0-d16d-4bcc-bb9e-218c0388c7a8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:29.370958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f641f0-d16d-4bcc-bb9e-218c0388c7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:29.370958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f641f0-d16d-4bcc-bb9e-218c0388c7a8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:29.372243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 554/1139] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:29.378494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:29.379057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:29.379057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:29.402680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:29.403954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:29.479872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 586/1140] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:29.750889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f641f0-d16d-4bcc-bb9e-218c0388c7a8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:29.855612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f641f0-d16d-4bcc-bb9e-218c0388c7a8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:29.890311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:29.890936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:29.891214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:29.891484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:29.902345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:29.907116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:29.908263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:29.909422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 555/1141] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:04:29.914422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:29.937807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:30.015122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a9e13-5b48-4638-a7c0-ba9ef54ca317 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:30.050656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:30.051262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:30.051540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:30.051810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:30.052466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:30.052765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:30.054168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:30.062208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 587/1142] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:30.068678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:30.091940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:30.169672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc7ba-2789-4080-80a1-4a5d8971eea1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:30.203654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:30.204278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:30.204556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:30.204929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] PATH_INFO: `/v3/users/374a2c795a3346269b427a975bc8f2ec` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:30.218061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] RBAC: Authorizing `identity:get_user(user_id=374a2c795a3346269b427a975bc8f2ec)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:30.219493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:30.221159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 556/1143] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:30 2022] GET /identity/v3/users/374a2c795a3346269b427a975bc8f2ec => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:30.227278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:30.311918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:30.328774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab73c3-6101-4885-8292-3bab7df7226c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:30.363629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:30.364236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:30.364632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:30.364780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:30.436002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:30.437607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:30.523260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:30.540029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 588/1144] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:30.546417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:30.628200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:30.648975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93817e-0836-4f92-9add-fab6b7759120 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:30.683936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:30.684549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:30.684825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:30.685096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] PATH_INFO: `/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:30.686229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] RBAC: Authorizing `identity:get_project(project_id=71c3fb65e59a49c58ad693692bf0b7b2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:30.687655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be2600-c867-4e49-a09a-779e54910172 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:30.689779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 557/1145] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:30 2022] GET /identity/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:30.695943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:30.832037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6050-08ad-46d6-829c-d22bbe417968 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:30.872119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:30.872683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:30.872925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:30.873196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:30.873994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=71c3fb65e59a49c58ad693692bf0b7b2, user.id=374a2c795a3346269b427a975bc8f2ec) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:30.874302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:30.875752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f583d6b-5c2c-4bc8-b472-a95d8093c2b8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:30.882050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 589/1146] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:30 2022] GET /identity/v3/role_assignments?user.id=374a2c795a3346269b427a975bc8f2ec&scope.project.id=71c3fb65e59a49c58ad693692bf0b7b2&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:32.835077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dae-81ad-409e-bb79-a2ac3dbfdeb3 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:32.835635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dae-81ad-409e-bb79-a2ac3dbfdeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:32.835635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dae-81ad-409e-bb79-a2ac3dbfdeb3 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:32.836928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 558/1147] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:32.842241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:32.842658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:32.842771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:32.860712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:32.861903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:32.907353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:32.932782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 590/1148] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:33.203268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7dae-81ad-409e-bb79-a2ac3dbfdeb3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:33.306962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7dae-81ad-409e-bb79-a2ac3dbfdeb3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:33.341368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:33.342023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:33.342297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:33.342562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:33.353276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:33.358025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:33.359181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:33.360161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 559/1149] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:33.365865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:33.465489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e7511-3f5f-4198-b3df-542b4ff5eca1 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:33.518253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:33.519243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:33.519714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:33.520186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:33.521370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:33.521931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:33.524278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:33.539679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 591/1150] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:33.545551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:33.555312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:33.648544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e304e8-7a53-494a-ac9b-f0d030b45bf1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:33.682271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:33.682901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:33.683181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:33.683455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] PATH_INFO: `/v3/users/374a2c795a3346269b427a975bc8f2ec` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:33.684593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] RBAC: Authorizing `identity:get_user(user_id=374a2c795a3346269b427a975bc8f2ec)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:33.686065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:33.687648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 560/1151] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:33 2022] GET /identity/v3/users/374a2c795a3346269b427a975bc8f2ec => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:33.694275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:33.738127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:33.797421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff0e8-f1f5-4a42-9297-0cac96b74ae1 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:33.831915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:33.832543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:33.832824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:33.833096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] PATH_INFO: `/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:33.834227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] RBAC: Authorizing `identity:get_project(project_id=71c3fb65e59a49c58ad693692bf0b7b2)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:33.835654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a443a-3350-4eec-9e2c-f119e8d69324 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:33.837281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 592/1152] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:33 2022] GET /identity/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:33.843511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:33.921436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:33.947714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64735a7-9818-4ade-94d0-b09a384f91c4 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:33.990976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:33.991542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:33.991817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:33.992088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] PATH_INFO: `/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2/users/374a2c795a3346269b427a975bc8f2ec/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:33.994314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] RBAC: Authorizing `identity:create_grant(project_id=71c3fb65e59a49c58ad693692bf0b7b2, user_id=374a2c795a3346269b427a975bc8f2ec, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:33.995741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7a707-659b-4a28-8f92-ecdd9db7b5ae None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:34.097925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 561/1153] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:04:33 2022] PUT /identity/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2/users/374a2c795a3346269b427a975bc8f2ec/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:04:36.059332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610116a3-8302-45c6-9e22-e3d67aa46602 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:36.059863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610116a3-8302-45c6-9e22-e3d67aa46602 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:36.059863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610116a3-8302-45c6-9e22-e3d67aa46602 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:36.061203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 593/1154] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:36.067063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdda994-1f64-4873-91d6-109587515aca None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:36.067505 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdda994-1f64-4873-91d6-109587515aca None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:36.067617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdda994-1f64-4873-91d6-109587515aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:36.088340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1cdda994-1f64-4873-91d6-109587515aca None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:36.089546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1cdda994-1f64-4873-91d6-109587515aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:36.158074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 562/1155] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:36.428287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610116a3-8302-45c6-9e22-e3d67aa46602 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:36.537125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610116a3-8302-45c6-9e22-e3d67aa46602 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:36.546615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:36.572377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:36.573003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:36.573277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:36.573544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:36.584426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:36.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:36.590779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:36.591732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 594/1156] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:36.597702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdda994-1f64-4873-91d6-109587515aca None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:36.663212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:36.701267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdda994-1f64-4873-91d6-109587515aca None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:36.737362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:36.738466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:36.738744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:36.739013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:36.739671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:36.739972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:36.741343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:36.750048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 563/1157] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:36.756819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:36.855781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd11c8-e12c-4f08-8d7a-e4ad858e40cb None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:36.890514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:36.891131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:36.891401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:36.891665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] PATH_INFO: `/v3/users/374a2c795a3346269b427a975bc8f2ec` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:36.892856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] RBAC: Authorizing `identity:get_user(user_id=374a2c795a3346269b427a975bc8f2ec)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:36.894465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:36.896017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 595/1158] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:36 2022] GET /identity/v3/users/374a2c795a3346269b427a975bc8f2ec => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:36.902142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:37.004871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cdc18-f51f-4e8a-a52f-c7cd1e608fc9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:37.030491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:37.038993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:37.042865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:37.042865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:37.042865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:37.107493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:37.108943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:37.208717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 564/1159] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:37.215033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:37.256463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:37.323429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52f3c7-aca1-435e-8d6e-3736d8b9951c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:37.359135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:37.359759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:37.360036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:37.360308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] PATH_INFO: `/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:37.362216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] RBAC: Authorizing `identity:get_project(project_id=71c3fb65e59a49c58ad693692bf0b7b2)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:37.363659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181f8a-fa9a-4b2b-a5a3-f6248ff9e6db None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:37.365275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 596/1160] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:37 2022] GET /identity/v3/projects/71c3fb65e59a49c58ad693692bf0b7b2 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:37.371776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:37.400575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:37.478777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178522f-d331-401d-8082-ecb123c7fac6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:37.513937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:37.514450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:37.514728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:37.514999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:37.515722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=71c3fb65e59a49c58ad693692bf0b7b2, user.id=374a2c795a3346269b427a975bc8f2ec) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:37.516026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:37.518164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb8a60-af86-411b-8de3-a9eeb4354edc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:37.524085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 565/1161] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:37 2022] GET /identity/v3/role_assignments?user.id=374a2c795a3346269b427a975bc8f2ec&scope.project.id=71c3fb65e59a49c58ad693692bf0b7b2&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:39.498425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b906-9135-4849-875e-ff7ee766c977 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:39.498964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b906-9135-4849-875e-ff7ee766c977 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:39.498964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7b906-9135-4849-875e-ff7ee766c977 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:39.500283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 597/1162] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:39.506321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:39.506784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:39.506874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:39.526101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:39.527303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:39.595327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 566/1163] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:39.865830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7b906-9135-4849-875e-ff7ee766c977 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:39.968472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7b906-9135-4849-875e-ff7ee766c977 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:39.971558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:40.003670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:40.004275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:40.004552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:40.004821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:40.073693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:40.075160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:40.175317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 598/1164] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:40.181321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:40.202694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:40.282204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5a3f9-7b25-4132-896e-9fbdbe34fa71 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:40.315614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:40.316225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:40.316502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:40.316768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:40.323400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:40.324833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-8fdd0a1c-4aad-41ca-afd3-4fb610e2dd9b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:40.336866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:40.338125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:40.339182 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 567/1165] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:04:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:40.344412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:40.347590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:40.444177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e716e-b315-4c0c-a15e-0aacbb56a878 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:40.479455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:40.480059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:40.480337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:40.480610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:40.481251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:40.481551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:40.482936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22cd62-a3d7-4b8e-98e8-dec8290025b7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:40.490796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 599/1166] 10.222.0.99 () {56 vars in 1115 bytes} [Fri Sep 2 21:04:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:42.420993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b117-036d-4a50-8897-f54e1c4ab327 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:42.421525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b117-036d-4a50-8897-f54e1c4ab327 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:42.421525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7b117-036d-4a50-8897-f54e1c4ab327 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:42.422943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 568/1167] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:42.428733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae72f-3daf-45bb-94c2-d7f07d479320 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:42.429177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae72f-3daf-45bb-94c2-d7f07d479320 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:42.429288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ae72f-3daf-45bb-94c2-d7f07d479320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:42.447944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4c1ae72f-3daf-45bb-94c2-d7f07d479320 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:42.449164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4c1ae72f-3daf-45bb-94c2-d7f07d479320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:42.515820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:42.522530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 600/1168] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:42.795377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7b117-036d-4a50-8897-f54e1c4ab327 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:42.881094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:42.901436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7b117-036d-4a50-8897-f54e1c4ab327 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:42.937586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:42.938271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:42.938583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:42.938857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:42.940084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:42.941453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefba3f-a619-4c1f-9698-cd2e0112e8c5 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:43.023134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 569/1169] 10.222.0.99 () {60 vars in 1138 bytes} [Fri Sep 2 21:04:42 2022] POST /identity/v3/domains => generated 267 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:04:44.989661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf717-c248-4a3d-acbe-8252a1681b79 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:44.990283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf717-c248-4a3d-acbe-8252a1681b79 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:44.990283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbf717-c248-4a3d-acbe-8252a1681b79 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:44.991571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 601/1170] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:44.996966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:44.997417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:44.997527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:45.016408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:45.017605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:45.083855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 570/1171] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:45.387558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbf717-c248-4a3d-acbe-8252a1681b79 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:45.501122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbf717-c248-4a3d-acbe-8252a1681b79 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:45.537950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563564e-81af-491b-8771-cad196aef935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:45.538709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563564e-81af-491b-8771-cad196aef935 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:45.538992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563564e-81af-491b-8771-cad196aef935 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:45.539357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563564e-81af-491b-8771-cad196aef935 None admin] PATH_INFO: `/v3/domains/c02472b286db4069ac029f57a509aa3c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:45.550778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563564e-81af-491b-8771-cad196aef935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c02472b286db4069ac029f57a509aa3c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:45.552237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563564e-81af-491b-8771-cad196aef935 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:45.553894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 602/1172] 10.222.0.99 () {56 vars in 1183 bytes} [Fri Sep 2 21:04:45 2022] GET /identity/v3/domains/c02472b286db4069ac029f57a509aa3c => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:45.559722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:45.628695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:45.659155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33c011-14a9-43b2-a6c1-5023969be681 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:45.700109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:45.700718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:45.700993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:45.701264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:45.701944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:45.703375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3aa33d-c887-4293-b226-4b67d4f59d18 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:45.811912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 571/1173] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:04:45 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:04:47.789530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73230e-96fb-4959-967e-488aea57bc50 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:47.790115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73230e-96fb-4959-967e-488aea57bc50 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:47.790115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73230e-96fb-4959-967e-488aea57bc50 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:47.791597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 603/1174] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:47.798243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:47.799008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:47.799008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:47.818782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:47.820036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:47.899303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 572/1175] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:04:48.183685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73230e-96fb-4959-967e-488aea57bc50 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:48.246853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:48.303858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73230e-96fb-4959-967e-488aea57bc50 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:48.346745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:48.347389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:48.347663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:48.347931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] PATH_INFO: `/v3/domains/c02472b286db4069ac029f57a509aa3c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:48.349095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c02472b286db4069ac029f57a509aa3c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:48.350577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d35f6-6c8c-42ee-b28a-6e842a184e54 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:48.352206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 604/1176] 10.222.0.99 () {56 vars in 1183 bytes} [Fri Sep 2 21:04:48 2022] GET /identity/v3/domains/c02472b286db4069ac029f57a509aa3c => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:48.358299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:48.416240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:48.466519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0c494-f0b1-4ec6-995f-b4ff7bd33d94 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:48.503876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:48.504485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:48.504760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:48.505028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:48.505748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:48.507167 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b6695-7897-44cf-a4d1-c49efa35f792 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:48.646483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 573/1177] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:04:48 2022] POST /identity/v3/users => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:04:50.691845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e60-756f-4e0b-b41d-f7640e08a9df None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:50.692412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e60-756f-4e0b-b41d-f7640e08a9df None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:50.692412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e60-756f-4e0b-b41d-f7640e08a9df None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:50.693735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 605/1178] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:50.699080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:50.699634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:50.699634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:50.719932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:50.721124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:50.794913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 574/1179] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:51.070124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f82e60-756f-4e0b-b41d-f7640e08a9df None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:51.181142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f82e60-756f-4e0b-b41d-f7640e08a9df None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:51.207765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:51.216201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:51.222360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:51.222360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:51.222360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:51.228980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:51.233846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:51.235115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:51.236459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 606/1180] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:51.241396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:51.285528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:51.347746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656dec9-e236-4e90-ae1e-5ac5a6c1d4a7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:51.382968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:51.390488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:51.396059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 575/1181] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:51.402412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:51.508917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0382d6-17f5-4d3c-9f35-38fdb7c9a0a4 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:51.543848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:51.544481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:51.544760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:51.545036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] PATH_INFO: `/v3/users/ef7670ce23e94eb2ad67a4989abe3622` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:51.559003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7670ce23e94eb2ad67a4989abe3622)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:51.560455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:51.562152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 607/1182] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:51 2022] GET /identity/v3/users/ef7670ce23e94eb2ad67a4989abe3622 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:51.569095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:51.668782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:51.671685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcd2f7-e6c7-4852-b70e-8980a5e7dd73 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:51.707675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:51.708330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:51.708541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:51.708812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:51.785427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:51.786908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:51.883238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:51.886858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 576/1183] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:51.892969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:51.940451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:51.995863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c3734-37d5-41db-91aa-25c41567df50 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:52.032586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:52.033210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:52.033487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:52.033781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] PATH_INFO: `/v3/projects/bbd247c6a1154381aefe40255051aa51` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:52.034885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=bbd247c6a1154381aefe40255051aa51)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:52.036301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4bdb0-6547-49f6-a749-a25cc5e3d3c0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:52.037951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 608/1184] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:51 2022] GET /identity/v3/projects/bbd247c6a1154381aefe40255051aa51 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:52.044144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:52.141917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5e0e-c7c9-4324-97ab-886ba154856e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:52.176723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:52.177326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:52.177603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:52.177905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:52.178623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=bbd247c6a1154381aefe40255051aa51, user.id=ef7670ce23e94eb2ad67a4989abe3622) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:52.178926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:52.180353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee6fe3-2dd3-44d4-9e73-fbfc89f9487c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:52.185803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 577/1185] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:52 2022] GET /identity/v3/role_assignments?user.id=ef7670ce23e94eb2ad67a4989abe3622&scope.project.id=bbd247c6a1154381aefe40255051aa51&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:54.160540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c69a4-7037-4f99-be83-8e2de991c95a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:54.161103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c69a4-7037-4f99-be83-8e2de991c95a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:54.161103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c69a4-7037-4f99-be83-8e2de991c95a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:54.162539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 609/1186] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:54.167862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:54.168276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:54.168401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:54.187577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:54.189386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:54.252045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:54.260676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 578/1187] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:04:54.546343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c69a4-7037-4f99-be83-8e2de991c95a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:54.655247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c69a4-7037-4f99-be83-8e2de991c95a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:54.670196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:54.692615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:54.694650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:54.694650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:54.694650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:54.705095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:54.709913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:54.711016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:04:54.712126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 610/1188] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:54.717141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:54.817143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90144bad-68a7-4575-b6c8-182bb89a3c79 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:54.849980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:54.850581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:54.850852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:54.851115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:54.851757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:54.852051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:54.853399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:54.861747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 579/1189] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:54.868126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:54.970341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177e232-5d31-46c6-bc36-cca567d25f81 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:55.004188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:55.004808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:55.005085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:55.005387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] PATH_INFO: `/v3/users/ef7670ce23e94eb2ad67a4989abe3622` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:55.006523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] RBAC: Authorizing `identity:get_user(user_id=ef7670ce23e94eb2ad67a4989abe3622)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:55.007951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:55.009514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 611/1190] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:54 2022] GET /identity/v3/users/ef7670ce23e94eb2ad67a4989abe3622 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:55.015471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:55.070535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:55.114056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be7e2-d6dc-41fc-95dd-d24613efd913 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:55.148296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:55.148917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:55.149195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:55.149463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] PATH_INFO: `/v3/projects/bbd247c6a1154381aefe40255051aa51` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:55.150605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] RBAC: Authorizing `identity:get_project(project_id=bbd247c6a1154381aefe40255051aa51)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:55.152033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70c5a-fc6d-4e3f-9468-00bed32bebc1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:55.153739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 580/1191] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:55 2022] GET /identity/v3/projects/bbd247c6a1154381aefe40255051aa51 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:55.160213 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:55.202346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:55.262020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9104bf-074a-4ecc-a6d6-71c5ee0bf86d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:55.295219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:55.295862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:55.296136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:55.296404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] PATH_INFO: `/v3/projects/bbd247c6a1154381aefe40255051aa51/users/ef7670ce23e94eb2ad67a4989abe3622/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:55.298603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbd247c6a1154381aefe40255051aa51, user_id=ef7670ce23e94eb2ad67a4989abe3622, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:55.300018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c66c3a-a80e-4a02-9f6e-beaedd1c58e1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:55.432011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 612/1192] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:04:55 2022] PUT /identity/v3/projects/bbd247c6a1154381aefe40255051aa51/users/ef7670ce23e94eb2ad67a4989abe3622/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:04:57.409110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7888def-0d0c-4173-bf16-a7a10f4a62f0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:57.409649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7888def-0d0c-4173-bf16-a7a10f4a62f0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:57.409649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7888def-0d0c-4173-bf16-a7a10f4a62f0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:57.410979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 581/1193] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:04:57.416320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:57.416728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:57.416846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:57.435372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:04:57.436546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:04:57.502822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 613/1194] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:04:57.771000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7888def-0d0c-4173-bf16-a7a10f4a62f0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:57.884273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7888def-0d0c-4173-bf16-a7a10f4a62f0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:57.912262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:57.922857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:57.923482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:57.923751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:57.925530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:57.934393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:04:57.939262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:04:57.940545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:04:57.941495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 582/1195] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:04:57.948484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:57.980380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:58.052234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031600f-5f74-498d-823e-65a331ff4110 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:58.087208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:58.087862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:58.088134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:58.088397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:58.089050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:58.089343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:58.090744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:58.099298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 614/1196] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:58.105207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:58.204871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b12df-db7f-4cd3-92b3-f2a67f38ec3a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:58.239950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:58.240591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:58.240863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:58.241133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] PATH_INFO: `/v3/users/ef7670ce23e94eb2ad67a4989abe3622` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:58.242458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7670ce23e94eb2ad67a4989abe3622)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:58.244159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:58.245808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 583/1197] 10.222.0.99 () {56 vars in 1177 bytes} [Fri Sep 2 21:04:58 2022] GET /identity/v3/users/ef7670ce23e94eb2ad67a4989abe3622 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:58.251997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:58.345370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:58.353720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7b65-fd6b-4fd7-a0d0-887e84aaf76a None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:58.388011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:58.388615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:58.388890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:58.389158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:58.456096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:58.457541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:58.545823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:58.555738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 615/1198] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:04:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:04:58.561804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:58.615048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:04:58.663719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e8dc-6140-4dd2-a7c1-68aa38ce0093 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:58.696166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:58.696898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:58.697268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:58.697626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] PATH_INFO: `/v3/projects/bbd247c6a1154381aefe40255051aa51` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:58.698913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=bbd247c6a1154381aefe40255051aa51)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:58.700420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f25c39-67eb-48fa-a978-e4d65033c5e3 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:58.702067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 584/1199] 10.222.0.99 () {56 vars in 1186 bytes} [Fri Sep 2 21:04:58 2022] GET /identity/v3/projects/bbd247c6a1154381aefe40255051aa51 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:04:58.709204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:04:58.811720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab5a9b-41d8-4c09-964c-e98b17328b7b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:04:58.847131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:04:58.847733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:04:58.848010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:04:58.848284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:04:58.848989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=bbd247c6a1154381aefe40255051aa51, user.id=ef7670ce23e94eb2ad67a4989abe3622) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:04:58.849295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:04:58.850756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e721d-54ad-4d84-a26f-8db4e6d7dc4e None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:04:58.856390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 616/1200] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:04:58 2022] GET /identity/v3/role_assignments?user.id=ef7670ce23e94eb2ad67a4989abe3622&scope.project.id=bbd247c6a1154381aefe40255051aa51&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:05:08.691796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=40 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:05:08.853882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=72 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:18.660946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad390f-dcd0-4ded-9e2a-a684cd1e64b6 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:18.661705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad390f-dcd0-4ded-9e2a-a684cd1e64b6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:18.661705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad390f-dcd0-4ded-9e2a-a684cd1e64b6 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:18.662877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 585/1201] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:18.669633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:18.670151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:18.670251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:18.698176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:18.699489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:18.770983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 617/1202] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:19.055308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad390f-dcd0-4ded-9e2a-a684cd1e64b6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:19.162192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad390f-dcd0-4ded-9e2a-a684cd1e64b6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:19.198159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:19.204225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:19.204225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:19.204225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:19.209939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:19.211358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:19.225572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:19.226919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:19.228244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:19.228244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:19.228244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:19.228244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 586/1203] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:19.232814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:19.332699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b52ff-4e3d-406c-98e8-2583319b27e4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:19.362429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:19.368517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:19.369127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:19.369405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:19.369678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:19.373124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:19.373124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:19.373124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7eda53-475c-4e81-8451-4eec62150e21 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:19.383826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 618/1204] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:19.390479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:19.444990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:19.498310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fc0ae-55d5-4599-9c37-ddddc6bd3878 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:19.532823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:19.533421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:19.533705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:19.534027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:19.534702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:19.536237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5257f-038d-49eb-8d86-4a3b6fc68182 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:19.658427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 587/1205] 10.222.0.99 () {60 vars in 1132 bytes} [Fri Sep 2 21:06:19 2022] POST /identity/v3/users => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:06:21.670420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d272a-47ce-4701-bad3-68dbc5028e2d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:21.670988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d272a-47ce-4701-bad3-68dbc5028e2d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:21.670988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d272a-47ce-4701-bad3-68dbc5028e2d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:21.672319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 619/1206] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:21.677415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:21.677864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:21.678044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:21.697571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:21.701812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:21.766503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 588/1207] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:22.034607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d272a-47ce-4701-bad3-68dbc5028e2d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.139369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d272a-47ce-4701-bad3-68dbc5028e2d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.173158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.173805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.174083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.174398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.185390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:22.190270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:22.191484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:22.192508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 620/1208] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:22.198219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.238753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:22.298384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb410-f4b7-4f41-aa41-d4e079abd565 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.331880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.334787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.334787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.334787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.334787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:22.334787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:22.335406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:22.343731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 589/1209] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:22.349481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.400784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:22.450679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b2830-d649-4d54-a40b-4f0d2fb10aa5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.485035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.489028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.489028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.489028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.496156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:22.497632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:22.511718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:22.516573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:22.517699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:22.517699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:22.517699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:22.517699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 621/1210] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:22.520386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.625495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:22.628422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5948ce-f65a-44e0-82c8-ba17579f5228 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.664453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.665053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.665355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.665620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.666342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:22.666643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:22.667979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:22.679125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 590/1211] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:22.685516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.786866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aa87f-7f8a-409c-adae-948d2e924cf6 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.788375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:22.820468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.821089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.821387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.821672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.829609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:22.831087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:22.843612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:22.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:22.845996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 622/1212] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:22.851254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:22.952887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edda2-2f8e-4409-b5b6-cc5bd13b00f0 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:22.986118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:22.986776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:22.987055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:22.987325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:22.988065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:22.988358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:22.989809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.007716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 591/1213] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:23.014922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:23.118993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101c44-c48d-4999-a77c-9af4e54136e1 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:23.154855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:23.160648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:23.160648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:23.160648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:23.186203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:23.224068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:23.225530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.325832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 623/1214] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:06:23.331824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:23.350771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:23.431050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29df14-1df0-49e2-8288-9e128884efaf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:23.464833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:23.465784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:23.466110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:23.466380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:23.476179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:23.477542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:23.493256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:23.494453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:23.495511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:23.495511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:23.495511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:23.495511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 592/1215] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:23.500976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:23.562435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:23.601422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e723d43-2bec-4a87-a5bd-cd4e28037208 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:23.637057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:23.638357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:23.638656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:23.645322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:23.645322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:23.645322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:23.645322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.652714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 624/1216] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:23.658593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:23.727933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:23.758778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f467e5-ae6f-413a-a930-1b8d0261f9b2 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:23.791650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:23.795062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:23.795062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:23.795062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:23.799946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:23.801383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:23.813123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:23.814249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:23.815415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 593/1217] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:23.820420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:23.935690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d466da-0fc8-40f5-9fdc-c3931e978ee9 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:23.958314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:23.970457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:23.971055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:23.971328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:23.971591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:23.972264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:23.972557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:23.973989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d4fe4-8baa-4d2d-8b9a-d9e4e0a6a8c2 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:23.988145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 625/1218] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:23.994688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:24.111826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf12f-d229-4fee-9001-c6c7f149ce00 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:24.154322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:24.162074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:24.162953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:24.163397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:24.163870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:24.164798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=9af4ca265da3485cb7cec625913274ff) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:24.165106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:24.168088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37938e3-3fd9-48dc-b21c-4caf84871574 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:24.176401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 594/1219] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:06:23 2022] GET /identity/v3/role_assignments?user.id=9af4ca265da3485cb7cec625913274ff&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:26.124604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016cfd-bd11-4169-8fde-829f063d8546 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:26.125167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016cfd-bd11-4169-8fde-829f063d8546 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:26.125167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24016cfd-bd11-4169-8fde-829f063d8546 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:26.126546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 626/1220] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:26.131747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b283-5a51-4112-b034-72b412081925 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:26.132144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b283-5a51-4112-b034-72b412081925 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:26.132290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9b283-5a51-4112-b034-72b412081925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:26.151273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-29e9b283-5a51-4112-b034-72b412081925 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:26.152570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-29e9b283-5a51-4112-b034-72b412081925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:26.221903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 595/1221] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:26.494812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24016cfd-bd11-4169-8fde-829f063d8546 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:26.598105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24016cfd-bd11-4169-8fde-829f063d8546 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:26.634821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:26.635462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:26.635740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:26.636055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:26.646998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:26.651907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:26.653005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:26.654230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 627/1222] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:26.659202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9b283-5a51-4112-b034-72b412081925 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:26.759569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9b283-5a51-4112-b034-72b412081925 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:26.783099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:26.794124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:26.794694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:26.794964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:26.795233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:26.795882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] RBAC: Adding query filter params (name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:26.796177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:26.797537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:26.805272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 596/1223] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:06:26.811205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:26.855741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:26.926185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb1f00-58fb-4921-91ff-7f465741b4d3 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:26.960728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:26.961405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:26.961760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:26.962048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:26.972946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:26.974416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:26.989705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:26.990907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:26.992511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:26.992511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:26.992511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:26.992511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 628/1224] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:26.996914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.093628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78846676-f2bf-49e7-94da-9e0bd98b443f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.128773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.129374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.129645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.129941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.130665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:27.130878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.132235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.145815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 597/1225] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:27.151630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.251731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:27.258296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccb05-8805-49f2-8a95-e7d7cff60909 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.292027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.296096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.296096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.296096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.301673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.303138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:27.315199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:27.316347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:27.317683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 629/1226] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:27.322451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.370383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:27.422754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33592f8-ec16-404b-97f2-1536b06f0ff1 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.456097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.462885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.477479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 598/1227] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:27.484068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.587497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146272-ec11-408e-8826-8fa70797b696 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.624178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.624724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.624998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.625266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.636661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.638314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:27.652483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:27.654112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:27.655275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:27.655275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:27.655275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:27.655275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 630/1228] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:27.660445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.761638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76918c-53a5-45d3-9d00-08a9047099ac None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.764285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:27.796993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.797591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.797889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.798159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.801622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:27.801622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.801622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.811989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 599/1229] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:27.817847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:27.828177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:27.919171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d792-999c-4ce6-889b-831e51453cac None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:27.954796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:27.955424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:27.955726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:27.955996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:27.962819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:27.968958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:27.976662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:27.977827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:27.978929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 631/1230] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:06:27.983978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:28.082902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d39332-0af2-4200-805f-3a941ee15c5a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:28.115643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:28.116237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:28.116768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:28.117039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:28.117707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:28.118027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:28.119444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05aa699-f39a-405d-ab8d-9a37b488fc4d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:28.131137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 600/1231] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:28.137956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:28.207782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:28.242783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb38c6-8d46-4be7-9a23-7b1d27f72b66 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:28.281162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:28.281859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:28.282139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:28.282410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/9af4ca265da3485cb7cec625913274ff/roles/b717bf6f9f9248cfbc4ecb89192a0f1f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:28.297372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=9af4ca265da3485cb7cec625913274ff, role_id=b717bf6f9f9248cfbc4ecb89192a0f1f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:28.298842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc03b80-7db5-43d5-bbd9-30c244f80b05 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:28.405511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 632/1232] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:06:28 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/9af4ca265da3485cb7cec625913274ff/roles/b717bf6f9f9248cfbc4ecb89192a0f1f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:06:30.344588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7398bce-ddec-4a65-b5b7-00c064ffb85b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:30.345135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7398bce-ddec-4a65-b5b7-00c064ffb85b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:30.345135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7398bce-ddec-4a65-b5b7-00c064ffb85b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:30.346510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 601/1233] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:30.351869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:30.352335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:30.352429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:30.371276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:30.373350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:30.442812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 633/1234] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 21:06:30.717656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7398bce-ddec-4a65-b5b7-00c064ffb85b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:30.743383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:30.819349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7398bce-ddec-4a65-b5b7-00c064ffb85b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:30.852704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:30.853318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:30.853589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:30.853882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:30.864477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:30.869369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:30.870517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:30.871606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 602/1235] 10.222.0.99 () {56 vars in 1102 bytes} [Fri Sep 2 21:06:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:30.877331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:30.924424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:30.976186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77699aa3-8b4a-4c84-9eb1-714890a281dd None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.011222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.011821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.012220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.012496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.013147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] RBAC: Adding query filter params (name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:31.013443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.014825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.022636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 634/1236] 10.222.0.99 () {56 vars in 1103 bytes} [Fri Sep 2 21:06:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:31.028705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:31.114069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:31.132362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b0fd-e90e-41cb-8637-ce6a173b6be6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.167765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.169451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.169757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.170072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.180940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.182412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:31.198131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:31.199343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:31.200438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:31.200438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:31.200438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:31.200438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 603/1237] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:31.205658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:31.308696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd40da-763c-400d-bf9d-98f966306678 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.317541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:31.342125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.342722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.342991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.343254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.343874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:31.344199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.345571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.357914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 635/1238] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:31.363619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:31.463529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee325a-6e9c-49c3-a4c0-7bb57736560b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.500463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.501071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.501344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.501605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.504327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:31.509363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.510808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:31.521734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:31.522936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:31.524186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 604/1239] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:31.529431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:31.632142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd157b3-37eb-4fc0-b27f-ef8a543c989b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.667323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.667934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.668210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.668602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.669253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:31.669549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.671172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.689873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 636/1240] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:31.696024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:31.796045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee4d43-af9b-4081-b7d9-34f997aff87e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:31.829645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:31.830391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:31.830747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:31.831103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:31.891129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:31.897441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:31.898912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:31.995760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 605/1241] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:06:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 21:06:32.002078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:32.029992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:32.101060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e859ca6-0f1e-4d7a-becd-ae8000c73a64 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:32.135908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:32.136565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:32.136847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:32.137746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:32.148022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:32.149502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:32.164238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:32.165285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:32.166292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:32.166292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:32.166292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:32.166292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 637/1242] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:32.171467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:32.270805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:32.276408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757b32-65b2-4514-8b76-f9d9dfaed91c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:32.311590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:32.312189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:32.312465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:32.312737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:32.313371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:32.313671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:32.315061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:32.326437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 606/1243] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:32.332205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:32.413363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:32.434323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f282c-518a-4637-bf5b-4695fb9c8814 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:32.469214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:32.469880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:32.470156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:32.470421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:32.476879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:32.478360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:32.489946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:32.491015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:32.492435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 638/1244] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:32.497493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:32.610563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6d365-995c-410f-9409-ee83394b3215 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:32.653764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:32.654366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:32.654723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:32.655008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:32.655672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:32.655965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:32.657373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e46b4-3cc0-4fc1-bb84-b1f9d87a4554 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:32.669840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 607/1245] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:32.675925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:32.780563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00751-52da-4d36-a48e-45180daeddc8 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:32.814901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:32.815566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:32.815844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:32.816114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:32.816828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] RBAC: Adding query filter params (role.id=b717bf6f9f9248cfbc4ecb89192a0f1f, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=9af4ca265da3485cb7cec625913274ff) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:32.817132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:32.818585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d8e82-2c69-4018-8bfc-cb215a64fbf5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:32.824393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 639/1246] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:06:32 2022] GET /identity/v3/role_assignments?user.id=9af4ca265da3485cb7cec625913274ff&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=b717bf6f9f9248cfbc4ecb89192a0f1f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:34.823143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d8542-4375-4abe-ba57-6d2756c9fadc None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:34.823670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d8542-4375-4abe-ba57-6d2756c9fadc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:34.823670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d8542-4375-4abe-ba57-6d2756c9fadc None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:34.825753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 608/1247] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:34.831176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:34.831595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:34.831714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:34.837985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:34.851230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:34.852430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:34.919169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 640/1248] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:35.183616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d8542-4375-4abe-ba57-6d2756c9fadc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:35.203159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:35.290782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d8542-4375-4abe-ba57-6d2756c9fadc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:35.330512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:35.331146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:35.331421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:35.331690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:35.342307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:35.347168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:35.348275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:35.349335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 609/1249] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:06:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:35.354331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:35.457239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105da48-9fe5-4594-9463-2eb61bb9e907 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:35.489347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:35.494829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:35.496037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:35.496307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:35.497169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:35.497969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:35.498315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:35.499744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:35.509166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 641/1250] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:35.514952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:35.601405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:35.632373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546cd64-eae9-4d6a-8358-e02b95a2fa9c None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:35.666081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:35.666700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:35.666978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:35.667248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:35.677891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:35.679333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:35.694879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:35.696508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:35.697927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:35.697927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:35.697927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:35.697927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 610/1251] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:35.702682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:35.805272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115207db-6f2f-4380-ab1e-04edd92e9fcd None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:35.838760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:35.839437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:35.839624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:35.839895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:35.841199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:35.841497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:35.842894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:35.854470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 642/1252] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:35.860836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:35.960937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad673c95-2201-4273-bb49-402cab33e76e None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:35.987349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:35.995201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:35.995821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:35.996096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:35.996364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.004749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.006233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:36.017658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:36.019083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:36.020379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 611/1253] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:36.024761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:36.081393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:36.125367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2f06-63ef-476a-af53-ebc17d1dc0ee None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:36.160138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:36.160741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:36.161022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:36.161294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.162042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:36.162346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.163780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:36.181467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 643/1254] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:36.188550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:36.290828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1796ece-e345-464b-9a9f-0d417ec5caf9 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:36.327537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:36.328137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:36.328409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:36.328673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.384566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:36.396869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.398551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:36.502824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 612/1255] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:06:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 21:06:36.509096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:36.611121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a9cf8-5530-4ba2-b159-fc51a1fa484d None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:36.625272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:36.650202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:36.650772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:36.651055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:36.651355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.663516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.667729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:36.679857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:36.681046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:36.682377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:36.682377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:36.682377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:36.682377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 644/1256] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 21:06:36.687007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:36.781162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:36.793174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f856f-df23-4522-9da2-03161aaede12 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:36.829217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:36.829854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:36.830132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:36.830399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.831034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:36.831331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.832686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:36.845170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 613/1257] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:36.850909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:36.952409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454eb3b-e122-4084-941b-343915475d04 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:36.985779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:36.986402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:36.986681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:36.986948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:36.993955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:36.995415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:37.008524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:37.009669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:37.016030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 645/1258] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:37.016633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:37.128624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ea18a-c01d-473f-87d2-0b5944b3224b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:37.161491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:37.162137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:37.162418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:37.162691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:37.163376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:37.163677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:37.165192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0684aef-be0b-4748-9784-0e350930401b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:37.177675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 614/1259] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:37.183906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:37.210870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:37.303127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da80-809a-42c4-8656-cab6f4f773e5 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:37.338365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:37.339026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:37.339233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:37.339507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:37.340735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=9af4ca265da3485cb7cec625913274ff) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:37.341043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:37.342501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a00bde-fbfa-4685-af1d-97856855ebc3 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:37.351331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 646/1260] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:06:37 2022] GET /identity/v3/role_assignments?user.id=9af4ca265da3485cb7cec625913274ff&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:39.245216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988fda5-3907-47f5-825d-d6d5268fa969 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:39.245801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988fda5-3907-47f5-825d-d6d5268fa969 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:39.245801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988fda5-3907-47f5-825d-d6d5268fa969 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:39.248244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 615/1261] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:39.262813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:39.263642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:39.263642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:39.281525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:39.282731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:39.349926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 647/1262] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:39.626627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988fda5-3907-47f5-825d-d6d5268fa969 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:39.638410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:39.733948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988fda5-3907-47f5-825d-d6d5268fa969 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:39.767895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:39.768511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:39.768781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:39.769045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:39.778904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:39.783672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:39.784880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:39.785794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 616/1263] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:39.790876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:39.811254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:39.891399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3b23-2df2-49a4-8669-a735c8e63ad3 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:39.924685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:39.925325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:39.925607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:39.925914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:39.926575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:39.926880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:39.928268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:39.937002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 648/1264] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:39.942841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.020317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:40.063001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eec28a-f451-47b6-9094-d555b57be8f6 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.100289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.100932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.101202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.101465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.111759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.113179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:40.127050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:40.128310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:40.129457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:40.129457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:40.129457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:40.129457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 617/1265] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:40.133984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.218146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:40.237602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07e850-a23e-4fdb-ab85-a9b802696715 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.272303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.272972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.273171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.273442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.274113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:40.274414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.275831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.288011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 649/1266] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:40.294800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.423855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46baa5-9e88-4dc9-90bb-999171e46e8f None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.442944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:40.456937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.457565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.457876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.465841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.466171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.467611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:40.478696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:40.479785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:40.481506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 618/1267] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:40.486371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.596061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15e224-89e7-40a5-b2c3-d64f6906390f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.630045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.630676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.630948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.631209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.631923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:40.632258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.633684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.643957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:40.651803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 650/1268] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:40.658562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.760592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0792da-5d2f-4a96-a1e5-bfc34357258b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.795196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.798035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.798035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.798035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.807134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.808633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:40.822876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:40.824042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:40.824967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:40.824967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:40.824967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:40.824967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 619/1269] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:40.830518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:40.932743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694d7a-777f-4bc5-82ca-9ab63dc22ba7 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:40.968571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:40.969264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:40.969494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:40.969793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:40.970424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:40.970727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:40.972084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:40.983906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 651/1270] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:40.989896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:41.023585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:41.089178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080665c6-fb16-45fe-9ff3-9cd881184ebf None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:41.125015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:41.125638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:41.125945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:41.126221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:41.133169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:41.134660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:41.146304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:41.147543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:41.148631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 620/1271] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:41.153452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:41.213736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:41.256379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154d8bf-b93f-4c45-b18a-0b5bf90a4e6b None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:41.292455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a6456-8577-4324-a317-920839c85175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:41.293091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6456-8577-4324-a317-920839c85175 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:41.293236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6456-8577-4324-a317-920839c85175 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:41.293499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a6456-8577-4324-a317-920839c85175 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:41.294253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a6456-8577-4324-a317-920839c85175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:41.295088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a6456-8577-4324-a317-920839c85175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:41.296501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a6456-8577-4324-a317-920839c85175 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:41.308527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 652/1272] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:41.314839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:41.408693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:41.417187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196eecd3-fe52-4c7f-8733-f4e68c34a465 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:41.451781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:41.452428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:41.452698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:41.452962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] PATH_INFO: `/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/9af4ca265da3485cb7cec625913274ff/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:41.454844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3c5c8aceb40a8be5c0864c4ec89ba, user_id=9af4ca265da3485cb7cec625913274ff, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:41.456357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25406c-e93a-4856-9e0e-e2f524320bcb None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:41.565870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 621/1273] 10.222.0.99 () {58 vars in 1439 bytes} [Fri Sep 2 21:06:41 2022] PUT /identity/v3/projects/34e3c5c8aceb40a8be5c0864c4ec89ba/users/9af4ca265da3485cb7cec625913274ff/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:06:43.506857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e6465-2c0d-447d-8b9b-a776ca683299 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:43.507390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e6465-2c0d-447d-8b9b-a776ca683299 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:43.507479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e6465-2c0d-447d-8b9b-a776ca683299 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:43.508745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 653/1274] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:43.514735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:43.515168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:43.515261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:43.533615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:43.534784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:43.601002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 622/1275] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:43.880507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e6465-2c0d-447d-8b9b-a776ca683299 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:43.992543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e6465-2c0d-447d-8b9b-a776ca683299 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.010623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:44.027690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.028320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.028599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.029401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.039435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 21:06:44.080209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:44.081362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:44.082479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 654/1276] 10.222.0.99 () {56 vars in 1096 bytes} [Fri Sep 2 21:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:44.083355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.139562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:44.157811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99d6c1-81ef-40ca-a6e6-a7a71a2a241a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.191465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.192066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.192346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.192615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.193264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:44.193562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:44.194951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:44.203032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 623/1277] 10.222.0.99 () {56 vars in 1099 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:44.208847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.318046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d115f0-e71a-4356-ada3-8f08076de3a0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.352328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.352952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.353243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.353514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.363426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:44.364899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:44.379652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:44.380796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:44.381917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:44.381917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:44.381917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:44.381917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 655/1278] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:44.386696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.490007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5033f9-9d18-425e-be2f-ddfade985990 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.523758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.524379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.524619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.524892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.525527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:44.525943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:44.527228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:44.539751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 624/1279] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:44.546167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.578218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:44.648890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec0743-9136-438c-af72-28cff94bd166 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.682369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.682981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.683259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.683528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.691204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:44.692624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 21:06:44.704353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:44.705514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:44.706589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 656/1280] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:44.711346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.715273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:44.810344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bdd7-8669-4431-b4ef-b56ca34f697b None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:44.846625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:44.847223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:44.847498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:44.847768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:44.848482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:44.848781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:44.850235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:44.866983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 625/1281] 10.222.0.99 () {56 vars in 1143 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:44.873316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:44.968553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:44.977876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc258bf-c827-43bd-a384-c710dd5ca487 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.011921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.012513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.012790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.013566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.082760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.084122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.172137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:45.182073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 657/1282] 10.222.0.99 () {58 vars in 1282 bytes} [Fri Sep 2 21:06:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 21:06:45.188741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:45.250132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:45.289730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984e57-c1e0-413a-919b-62692a0e1170 None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.328326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.328950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.329224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.329489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.339469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.340837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 21:06:45.356019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:45.357206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 21:06:45.358529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 21:06:45.358529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 21:06:45.358529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:45.358529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 626/1283] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:45.362890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:45.465814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747121f-a100-4e33-a04a-f3be70d9bd7f None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.502806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.503397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.503671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.503945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.504580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:45.504870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.506258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.517816 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 658/1284] 10.222.0.99 () {56 vars in 1109 bytes} [Fri Sep 2 21:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:45.523477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:45.625074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e86fe8-d33a-474a-99d5-e88654446ecd None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.638692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:45.658783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.659539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.659795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.660060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.666750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.668449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 21:06:45.680146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:45.681772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: ERROR keystone.server.flask.application Sep 02 21:06:45.682899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 627/1285] 10.222.0.99 () {56 vars in 1111 bytes} [Fri Sep 2 21:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:45.688384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:45.738628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:45.790937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9290d-9db5-42a1-a5c5-bb579a837554 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.823972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.824565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.824837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.825101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.825850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:45.826146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.827561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2905e6-95a0-490d-b04d-e403e6b3ad67 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.839285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 659/1286] 10.222.0.99 () {56 vars in 1148 bytes} [Fri Sep 2 21:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:45.846532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:45.945936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ec6df-7878-48a1-882a-f5b9f168596a None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:45.979673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:45.980264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:45.980586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:45.980865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:45.981617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] RBAC: Adding query filter params (role.id=23a6ea4d1c1648259ca786cd5becc2fe, scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba, user.id=9af4ca265da3485cb7cec625913274ff) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:45.982611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:45.984105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7dd8b-d7df-4aba-873e-7f6d8d3f3e1d None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:45.990442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 628/1287] 10.222.0.99 () {56 vars in 1374 bytes} [Fri Sep 2 21:06:45 2022] GET /identity/v3/role_assignments?user.id=9af4ca265da3485cb7cec625913274ff&scope.project.id=34e3c5c8aceb40a8be5c0864c4ec89ba&role.id=23a6ea4d1c1648259ca786cd5becc2fe => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:47.954211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737af-9689-4fc0-a06e-3a65466f171e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:47.954782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737af-9689-4fc0-a06e-3a65466f171e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:47.954782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e737af-9689-4fc0-a06e-3a65466f171e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:47.956001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 660/1288] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:47.961219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:47.961617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:47.961801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:47.980457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:47.981638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:48.011161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:48.048270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 629/1289] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:48.318774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e737af-9689-4fc0-a06e-3a65466f171e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:48.420356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e737af-9689-4fc0-a06e-3a65466f171e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:48.432541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:48.465706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:48.468447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:48.468447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:48.468447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:48.468447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:48.468927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:48.477649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:48.479027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:48.480140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 661/1290] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:06:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:48.485595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:48.587418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0e964-ea48-4984-90ca-758980b9d578 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:48.622132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:48.623410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:48.623694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:48.624051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:48.624682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:48.624982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:48.626375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d011e-7e02-4e1c-b371-63330f890959 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:48.631260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 630/1291] 10.222.0.99 () {56 vars in 1116 bytes} [Fri Sep 2 21:06:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:48.637193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:48.737822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476c3e2-4c2d-47a4-99da-d25739fe806c None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:48.771662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:48.772260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:48.772539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:48.772813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] PATH_INFO: `/v3/services` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:48.773435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] RBAC: Adding query filter params (type=placement) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:48.773757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:48.775119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c85e09-ec6c-4fbd-969f-902f9b4a00be None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:48.779605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 662/1292] 10.222.0.99 () {56 vars in 1116 bytes} [Fri Sep 2 21:06:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:50.740189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75e66-9bd1-4d5a-aa7e-c3525dae59ba None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:50.740767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75e66-9bd1-4d5a-aa7e-c3525dae59ba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:50.740767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75e66-9bd1-4d5a-aa7e-c3525dae59ba None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:50.742138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 631/1293] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:50.747867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80bdf-f088-4221-909b-651b1bdbea3d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:50.748941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80bdf-f088-4221-909b-651b1bdbea3d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:50.748941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80bdf-f088-4221-909b-651b1bdbea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:50.767282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-40a80bdf-f088-4221-909b-651b1bdbea3d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:50.768468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-40a80bdf-f088-4221-909b-651b1bdbea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:50.798319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:50.835817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 663/1294] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:51.118106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be75e66-9bd1-4d5a-aa7e-c3525dae59ba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:51.150294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:51.224498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be75e66-9bd1-4d5a-aa7e-c3525dae59ba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:51.263925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:51.264553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:51.264829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:51.265097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:51.265765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:51.267178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02322a-2213-4028-874f-8e9de86be988 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:51.374732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 632/1295] 10.222.0.99 () {60 vars in 1141 bytes} [Fri Sep 2 21:06:51 2022] POST /identity/v3/services => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:06:53.308535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f354-b733-4550-817f-a6f1425989fb None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:53.309102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f354-b733-4550-817f-a6f1425989fb None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:53.309102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663f354-b733-4550-817f-a6f1425989fb None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:53.311094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 664/1296] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:53.316672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:53.317158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:53.317259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:53.336044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:53.337224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:53.412914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 633/1297] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:53.683099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f354-b733-4550-817f-a6f1425989fb None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:53.784157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663f354-b733-4550-817f-a6f1425989fb None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:53.818364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:53.819031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:53.819303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:53.819567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:53.820133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:53.821474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:53.830532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:53.831562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:53.832422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 665/1298] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:06:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:53.837687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:53.855694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:53.936925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61268054-a604-4e92-b6e5-14bf8622394f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:53.970894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:53.971499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:53.971769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:53.972038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:53.972728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:53.972956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:53.974353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:53.979608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 634/1299] 10.222.0.99 () {56 vars in 1116 bytes} [Fri Sep 2 21:06:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:53.986905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:54.021244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:54.089912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49d7e3-c3d8-4b47-b243-310b55ab06d0 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:54.125843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:54.126445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:54.126722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:54.126995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:54.127681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] RBAC: Adding query filter params (interface=public, service_id=4953e19a296e4974b0ad56baaade18ea, region_id=RegionOne) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:54.127981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:54.129342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a146-faa5-4def-85b5-ec54b1a2d13a None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:54.134454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 666/1300] 10.222.0.99 () {56 vars in 1251 bytes} [Fri Sep 2 21:06:53 2022] GET /identity/v3/endpoints?service_id=4953e19a296e4974b0ad56baaade18ea&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:06:54.140565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:54.223940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:54.242841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d58d9-9939-4c8b-a3d9-b433fc8b89fc None admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:54.281813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:54.282438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:54.282720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:54.282994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:54.283600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:54.284982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce026d-5634-41eb-94b2-e3688a95ae91 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:54.292010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 635/1301] 10.222.0.99 () {56 vars in 1087 bytes} [Fri Sep 2 21:06:54 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:06:56.362574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6c16-8c0c-47d0-bacb-fae119e20281 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:56.363200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6c16-8c0c-47d0-bacb-fae119e20281 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:56.363200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c6c16-8c0c-47d0-bacb-fae119e20281 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:56.365076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 667/1302] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:06:56.370637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:56.371255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:56.371255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:56.390544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:06:56.391751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:06:56.459680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 636/1303] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:06:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 21:06:56.746292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6c16-8c0c-47d0-bacb-fae119e20281 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:56.858688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:56.866595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c6c16-8c0c-47d0-bacb-fae119e20281 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:56.908051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:56.908689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:56.908969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:56.909239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:56.909853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:56.911218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 21:06:56.921821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 21:06:56.923100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application with Lock( Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self._enter() Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application return self.creator() Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application created_value = creator( Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: ERROR keystone.server.flask.application Sep 02 21:06:56.924109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 668/1304] 10.222.0.99 () {56 vars in 1117 bytes} [Fri Sep 2 21:06:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 21:06:56.929042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:57.029874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:06:57.053584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffb0aa-141d-4a21-9cc7-e08d9f0aa9e3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:57.109269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:57.110312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:57.110718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:57.111201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] PATH_INFO: `/v3/services` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:57.112315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:06:57.112714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:57.115226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0cff3-531b-4d6c-b06b-20f0f26eb5c0 None admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:57.130511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 637/1305] 10.222.0.99 () {56 vars in 1116 bytes} [Fri Sep 2 21:06:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:06:57.130738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:06:57.273097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa342ea-9968-443f-82b8-2e805518e785 None admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:06:57.320058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:06:57.320058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:06:57.320058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:06:57.320510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:06:57.320898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:06:57.324347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e3512-2306-497d-8243-4ec00d1ca78f None admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:06:57.440026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 669/1306] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:06:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:07:07.159878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=30 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:07:07.430784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:31.318672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2cdf-4eb2-4646-9967-9e01bf5983d8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:31.319252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2cdf-4eb2-4646-9967-9e01bf5983d8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:31.319601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2cdf-4eb2-4646-9967-9e01bf5983d8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:31.326405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 638/1307] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:31.330793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb76b92-39fa-4b52-a871-b0d245d0f77d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:31.331388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb76b92-39fa-4b52-a871-b0d245d0f77d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:31.331726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb76b92-39fa-4b52-a871-b0d245d0f77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:31.502804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2cb76b92-39fa-4b52-a871-b0d245d0f77d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:31.504952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2cb76b92-39fa-4b52-a871-b0d245d0f77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:31.790327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 670/1308] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:32.302767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bd153-f90d-4118-a58f-c28395342061 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:32.303868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bd153-f90d-4118-a58f-c28395342061 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:32.304551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bd153-f90d-4118-a58f-c28395342061 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:32.308289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 639/1309] 10.222.0.99 () {54 vars in 965 bytes} [Fri Sep 2 21:15:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:32.320482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6ad59-0f2f-4d55-b8c4-9375f18bef77 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:32.321281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6ad59-0f2f-4d55-b8c4-9375f18bef77 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:32.322275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6ad59-0f2f-4d55-b8c4-9375f18bef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:32.568039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-aec6ad59-0f2f-4d55-b8c4-9375f18bef77 None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:32.570103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-aec6ad59-0f2f-4d55-b8c4-9375f18bef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:32.628252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:32.722113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bd153-f90d-4118-a58f-c28395342061 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:32.724646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-208bd153-f90d-4118-a58f-c28395342061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:32.735336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 671/1310] 10.222.0.99 () {58 vars in 1062 bytes} [Fri Sep 2 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:32.893748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bd153-f90d-4118-a58f-c28395342061 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:33.042664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:33.044152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:33.044926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:33.047113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:33.051649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:33.195976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:33.348465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:33.351694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff5129-cf8f-431d-a7a9-cce56f8db6d8 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:33.409512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 640/1311] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:15:36.144656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:36.145250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:36.145250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:36.146791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 672/1312] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:36.156818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9769d-b21c-4c2d-b258-2e5311b7282b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:36.157624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9769d-b21c-4c2d-b258-2e5311b7282b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:36.157624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9769d-b21c-4c2d-b258-2e5311b7282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:36.285553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-25a9769d-b21c-4c2d-b258-2e5311b7282b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:36.286893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-25a9769d-b21c-4c2d-b258-2e5311b7282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:36.445374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 641/1313] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:15:36.754822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:36.756467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:36.895058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61004e2-bfed-44dc-b695-b0e72234d957 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:36.987020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:36.990116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:36.990740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:36.991015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:36.991281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:36.993345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:37.175429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:37.178130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebf8d6-b6a2-48ce-baad-97d9fe989c66 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:37.201869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 673/1314] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:15:39.814790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:39.815428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:39.815428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:39.816868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 642/1315] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:39.823074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967f89-a19b-446e-945a-146c6848499c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:39.823660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967f89-a19b-446e-945a-146c6848499c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:39.823907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1967f89-a19b-446e-945a-146c6848499c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:39.906602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d1967f89-a19b-446e-945a-146c6848499c None None] MFA Rules not processed for user `25a810e91f074496a4fe262fab43ea53`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:39.907745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d1967f89-a19b-446e-945a-146c6848499c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:39.971887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:40.035132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 674/1316] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:15:40.324892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:40.327444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:40.344017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:40.465986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b6d44-94ec-4575-9e2b-e02ae7b84a90 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:40.555955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:40.556578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:40.556852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:40.557121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:40.558912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:40.637765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:40.695282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:40.696859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8582-ef53-4594-8360-0c2b9d4b08db service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:40.718253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 643/1317] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:15:43.783425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:43.784155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:43.784155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:43.785497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 675/1318] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:43.791792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed9ec-6d73-49c0-b221-d191b2d8c5b9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:43.792402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed9ec-6d73-49c0-b221-d191b2d8c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:43.792402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed9ec-6d73-49c0-b221-d191b2d8c5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:43.876333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-691ed9ec-6d73-49c0-b221-d191b2d8c5b9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:43.877533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-691ed9ec-6d73-49c0-b221-d191b2d8c5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:43.994791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 644/1319] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:44.286827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:44.288424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:44.418252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d78773-fc4f-4b10-bc95-734f23b6079a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:44.511022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:44.511642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:44.511916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:44.512186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:44.514724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:44.523554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:44.651012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:44.652457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60933ca6-1fd1-4e8a-a56d-176205177467 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:44.673795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 676/1320] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:15:48.167312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c47c-a6bc-4ddc-946a-0d016601269a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:48.177306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c47c-a6bc-4ddc-946a-0d016601269a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:48.177306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c47c-a6bc-4ddc-946a-0d016601269a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:48.177306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 645/1321] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:48.177801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f1984-c1f5-4099-9d11-81a423813936 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:48.177801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f1984-c1f5-4099-9d11-81a423813936 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:48.177801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f1984-c1f5-4099-9d11-81a423813936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:48.260602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-713f1984-c1f5-4099-9d11-81a423813936 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:48.261908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-713f1984-c1f5-4099-9d11-81a423813936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:48.335229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:48.390440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 677/1322] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:48.706287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:48.706717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:48.707121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:48.709642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 646/1323] 10.222.0.99 () {54 vars in 965 bytes} [Fri Sep 2 21:15:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:48.718151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2949d2-f1e3-4252-8081-3f6ef3cc4ff8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:48.718637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2949d2-f1e3-4252-8081-3f6ef3cc4ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:48.719113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2949d2-f1e3-4252-8081-3f6ef3cc4ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:48.830962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0c2949d2-f1e3-4252-8081-3f6ef3cc4ff8 None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:48.832322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0c2949d2-f1e3-4252-8081-3f6ef3cc4ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:48.940730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 678/1324] 10.222.0.99 () {58 vars in 1062 bytes} [Fri Sep 2 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:48.954075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:48.955886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:48.985673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:49.117786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9521ef-da2b-4bfa-9b28-fbea36a4a6ad None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:49.229575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:49.230580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:49.230726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:49.231156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:49.234480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:49.345626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:49.404984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:49.407254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6cb2a-0216-4122-aca7-e5179e941e2d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:49.432058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 647/1325] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:15:52.481896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:52.482621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:52.482750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:52.490200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 679/1326] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:52.494459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc49ad-5fdd-4a05-9296-b5bb259fc09e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:52.495052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc49ad-5fdd-4a05-9296-b5bb259fc09e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:52.495387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc49ad-5fdd-4a05-9296-b5bb259fc09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:52.609967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-96bc49ad-5fdd-4a05-9296-b5bb259fc09e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:52.612203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96bc49ad-5fdd-4a05-9296-b5bb259fc09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:52.808970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 648/1327] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:53.135507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:53.137572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:53.225312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:53.321433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f8e32-284b-4c12-bc56-36aaf67ef74f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:53.463051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:53.463676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:53.463980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:53.464245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:53.467638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:53.657131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:53.660312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:53.662888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5614c4c-8d5c-4b32-b56e-cc7e3c00aca8 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:53.697427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 680/1328] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:15:56.091743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:56.092598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:56.092598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:56.093833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 649/1329] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:15:56.101612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59754-16cc-40fb-95f5-cece1dc85669 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:56.102527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59754-16cc-40fb-95f5-cece1dc85669 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:56.102682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e59754-16cc-40fb-95f5-cece1dc85669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:56.228616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a8e59754-16cc-40fb-95f5-cece1dc85669 None None] MFA Rules not processed for user `25a810e91f074496a4fe262fab43ea53`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:15:56.230684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a8e59754-16cc-40fb-95f5-cece1dc85669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:56.391184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 681/1330] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:15:56.920621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:15:56.923664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:57.000811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:57.110947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e57db-ce5e-4a97-8051-25ab4992fd4e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:15:57.263391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:15:57.264353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:15:57.264737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:15:57.265007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:15:57.267083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:15:57.386470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:15:57.446173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:15:57.449051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89765c1-d8fc-43cc-8101-4f5374aff985 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:15:57.480805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 650/1331] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:00.313814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:00.314399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:00.314399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:00.315754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 682/1332] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:16:00.321890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c335c-7d0d-4d5f-86d9-1597dbfd8382 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:00.322726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c335c-7d0d-4d5f-86d9-1597dbfd8382 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:00.322884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c335c-7d0d-4d5f-86d9-1597dbfd8382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:00.411740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-638c335c-7d0d-4d5f-86d9-1597dbfd8382 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:00.412900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-638c335c-7d0d-4d5f-86d9-1597dbfd8382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:00.534173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 651/1333] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:00.839354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:00.842093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:00.930791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:00.978158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8027c0-b668-484a-a40b-5206eedf25c4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:01.070481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:01.071189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:01.071521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:01.071854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:01.073757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:01.258981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:01.260881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9f719-80f3-4a2b-abdc-f7bccf57aca5 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:01.278861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:01.288868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 683/1334] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:03.442736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:03.443635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:03.443635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:03.444650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 652/1335] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:16:03.451512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32e89-1b98-4141-99ba-00fa527bf927 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:03.452099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32e89-1b98-4141-99ba-00fa527bf927 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:03.452099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32e89-1b98-4141-99ba-00fa527bf927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:03.530894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-daf32e89-1b98-4141-99ba-00fa527bf927 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:03.532161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-daf32e89-1b98-4141-99ba-00fa527bf927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:03.647780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 684/1336] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:03.952365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:03.955620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:04.029099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:04.110836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32fbc8-bef4-4401-ab89-c29d0a2396a9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:04.208399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:04.208900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:04.209172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:04.209440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:04.211143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:04.354832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:04.401773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:04.406065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd1e0c-534b-40ae-8637-f6648abae04c service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:04.439645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 653/1337] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:07.394879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741798-fed4-4535-af45-7097001452a7 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:07.395507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741798-fed4-4535-af45-7097001452a7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:07.395507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741798-fed4-4535-af45-7097001452a7 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:07.396948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 685/1338] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:16:07.402948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cdfd4-2693-4d58-a756-9107f070b3cc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:07.403539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cdfd4-2693-4d58-a756-9107f070b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:07.403539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cdfd4-2693-4d58-a756-9107f070b3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:07.481781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4e1cdfd4-2693-4d58-a756-9107f070b3cc None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:07.483210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e1cdfd4-2693-4d58-a756-9107f070b3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:07.604285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 654/1339] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:07.908168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741798-fed4-4535-af45-7097001452a7 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:07.909747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8f741798-fed4-4535-af45-7097001452a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:08.004985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:08.039034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741798-fed4-4535-af45-7097001452a7 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:08.132632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:08.133303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:08.133851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:08.141354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:08.141354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:08.281459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:08.282976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd3aa6-7b1c-4ff8-9447-4c82ae57a386 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:08.304623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 686/1340] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:13.634011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:13.634776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:13.635089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:13.638181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 655/1341] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:13.645250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbb53a-a444-4680-a963-123582c946a5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:13.646102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbb53a-a444-4680-a963-123582c946a5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:13.646102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbb53a-a444-4680-a963-123582c946a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:13.652807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:13.747370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-16fbb53a-a444-4680-a963-123582c946a5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:13.749280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-16fbb53a-a444-4680-a963-123582c946a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:13.887865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 687/1342] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:14.184211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:14.186300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:14.247544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:14.339471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88c085-6ae4-4267-9f72-8a875578ef82 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:14.459844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:14.460835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:14.461316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:14.461934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:14.465771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:14.638589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:14.664251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:14.665925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca41cff-b6a8-4316-ad04-bc5ddb62bc20 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:14.695041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 656/1343] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:18.028169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:18.028883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:18.028883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:18.030259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 688/1344] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:18.036958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec198-2116-4126-b180-2379a9f76ab5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:18.043411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec198-2116-4126-b180-2379a9f76ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:18.043411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec198-2116-4126-b180-2379a9f76ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:18.129667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a32ec198-2116-4126-b180-2379a9f76ab5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:18.131739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a32ec198-2116-4126-b180-2379a9f76ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:18.294641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 657/1345] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:18.847274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:18.849674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:18.968779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:18.989739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaa21d-9f82-4eb7-bc6f-9aaef87bf415 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:19.086993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:19.087888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:19.087888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:19.088237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:19.089892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:19.259345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:19.261063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356891d-fd46-4922-ac75-7db4c85885db service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:19.285198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 689/1346] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:24.939674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:24.940518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:24.940518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:24.941645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 658/1347] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:24.948112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bd909-30f8-45ad-9fca-093a933e4e5d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:24.949106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bd909-30f8-45ad-9fca-093a933e4e5d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:24.950277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bd909-30f8-45ad-9fca-093a933e4e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:24.976489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:25.076701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-609bd909-30f8-45ad-9fca-093a933e4e5d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:25.078820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-609bd909-30f8-45ad-9fca-093a933e4e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:25.243147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 690/1348] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:25.561300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:25.564300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:25.672409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:25.747359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c775ce0-744a-4f64-9b25-32d834a6bdcd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:25.849919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:25.849919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:25.849919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:25.849919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:25.850573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:26.030998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:26.035883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:26.037576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974badae-6b6f-4475-9cad-dbbfb04a1f99 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:26.070074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 659/1349] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:31.898595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1ae-3004-4ba2-a731-77f12d97a68d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:31.899557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1ae-3004-4ba2-a731-77f12d97a68d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:31.899557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6af1ae-3004-4ba2-a731-77f12d97a68d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:31.900962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 691/1350] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:31.908791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb399-0bae-4295-8fb7-c5cc8a95778c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:31.909634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb399-0bae-4295-8fb7-c5cc8a95778c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:31.909634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb399-0bae-4295-8fb7-c5cc8a95778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:32.071874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7a8eb399-0bae-4295-8fb7-c5cc8a95778c None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:32.073641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a8eb399-0bae-4295-8fb7-c5cc8a95778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:32.278993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 660/1351] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:16:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:35.242138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=59 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:37.995252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3ca1c-99b5-4e49-b40b-b62d6f0fc41b None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:37.995930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3ca1c-99b5-4e49-b40b-b62d6f0fc41b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:37.995930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3ca1c-99b5-4e49-b40b-b62d6f0fc41b None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:37.997682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 692/1352] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:16:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:38.005088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122dad-4e2d-4b40-a249-eb4cc7c684ed None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:38.005608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122dad-4e2d-4b40-a249-eb4cc7c684ed None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:38.005755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122dad-4e2d-4b40-a249-eb4cc7c684ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:38.071420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:38.102868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5b122dad-4e2d-4b40-a249-eb4cc7c684ed None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:38.105337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b122dad-4e2d-4b40-a249-eb4cc7c684ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:38.318586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 661/1353] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:40.207880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240c65-7002-408a-a813-947a4e9baa00 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:40.208457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240c65-7002-408a-a813-947a4e9baa00 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:40.208556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97240c65-7002-408a-a813-947a4e9baa00 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:40.210129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 693/1354] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:40.216822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba5bce-9ffb-4e5d-bee3-47fdf1d76819 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:40.216822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba5bce-9ffb-4e5d-bee3-47fdf1d76819 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:40.218506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba5bce-9ffb-4e5d-bee3-47fdf1d76819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:40.321301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e4ba5bce-9ffb-4e5d-bee3-47fdf1d76819 None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:40.322775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e4ba5bce-9ffb-4e5d-bee3-47fdf1d76819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:40.389100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:40.505946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 662/1355] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:44.434475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0b00-d68e-420f-a680-b93597aaec94 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:44.435348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0b00-d68e-420f-a680-b93597aaec94 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:44.435536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c0b00-d68e-420f-a680-b93597aaec94 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:44.437107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 694/1356] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:44.444285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0f5c-d9f3-4b29-b504-20e4eb941c37 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:44.444828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0f5c-d9f3-4b29-b504-20e4eb941c37 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:44.445335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f0f5c-d9f3-4b29-b504-20e4eb941c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:44.603287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c53f0f5c-d9f3-4b29-b504-20e4eb941c37 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:44.604385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c53f0f5c-d9f3-4b29-b504-20e4eb941c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:44.762200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 663/1357] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:45.127126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:45.127935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:45.128529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:45.131485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 695/1358] 10.222.0.99 () {54 vars in 962 bytes} [Fri Sep 2 21:16:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:45.153030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098432-2c55-4838-bc20-bd6d9e81a33a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:45.153517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098432-2c55-4838-bc20-bd6d9e81a33a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:45.154313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098432-2c55-4838-bc20-bd6d9e81a33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:45.187148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:45.317764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8d098432-2c55-4838-bc20-bd6d9e81a33a None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:45.320833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8d098432-2c55-4838-bc20-bd6d9e81a33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:45.525156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 664/1359] 10.222.0.99 () {58 vars in 1059 bytes} [Fri Sep 2 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:45.536234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:45.538000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:45.823976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f4e1b-4091-4897-b958-81817b9570c8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:45.939091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:45.995364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:45.995997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:45.996282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:45.996540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:45.998615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:46.163744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:46.166143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750d5fc-d8e8-46ab-973d-12c23d9df6d1 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:46.204611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 696/1360] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:16:49.004427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:49.005026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:49.005405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:49.008264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 665/1361] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:49.017912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87bd6-d574-45b4-9f3c-0070dd8bce47 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:49.018804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87bd6-d574-45b4-9f3c-0070dd8bce47 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:49.019147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f87bd6-d574-45b4-9f3c-0070dd8bce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:49.159853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b0f87bd6-d574-45b4-9f3c-0070dd8bce47 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:49.160832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b0f87bd6-d574-45b4-9f3c-0070dd8bce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:49.247377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:49.337206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 697/1362] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:49.672945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:49.674574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:49.777954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:49.901900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302814b6-8945-4e34-8d8f-ad60635109ed None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:50.048389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:50.049063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:50.049197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:50.049466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:50.053782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:50.079911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:50.162693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:50.164334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7444149-e2a3-4e09-a42b-17a3ddacfe3c service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:50.187370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 666/1363] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:52.826151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c592e0-c9a5-4600-b1ba-49381e52cf32 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:52.826663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c592e0-c9a5-4600-b1ba-49381e52cf32 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:52.826663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c592e0-c9a5-4600-b1ba-49381e52cf32 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:52.828302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 698/1364] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:52.835702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34860-e2b6-4c7d-aedc-b85f997222f6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:52.836263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34860-e2b6-4c7d-aedc-b85f997222f6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:52.838472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e34860-e2b6-4c7d-aedc-b85f997222f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:52.955657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-71e34860-e2b6-4c7d-aedc-b85f997222f6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:52.957532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-71e34860-e2b6-4c7d-aedc-b85f997222f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:53.169887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:53.174633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 667/1365] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:53.502713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:53.503300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:53.503863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:53.507821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 699/1366] 10.222.0.99 () {54 vars in 962 bytes} [Fri Sep 2 21:16:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:53.515317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582874-2e72-43d8-a517-f02e752782d1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:53.515917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582874-2e72-43d8-a517-f02e752782d1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:53.516692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae582874-2e72-43d8-a517-f02e752782d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:53.694194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ae582874-2e72-43d8-a517-f02e752782d1 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:53.696316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ae582874-2e72-43d8-a517-f02e752782d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:53.903589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 668/1367] 10.222.0.99 () {58 vars in 1059 bytes} [Fri Sep 2 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:53.912933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:53.914533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:54.160638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a4559-bc0d-4487-b1ed-11126f0e7a79 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:54.189775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:54.350995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:54.351966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:54.352403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:54.352807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:54.356457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:54.477121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:54.479391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c637a-8295-4331-9ad2-4ebe304396d4 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:54.501427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 700/1368] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:55.755965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cde495-c3f8-49c4-9485-64392d5dec7a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:55.756840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cde495-c3f8-49c4-9485-64392d5dec7a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:55.756840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cde495-c3f8-49c4-9485-64392d5dec7a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:55.758381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 669/1369] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:16:55.765773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ce29b-bdcd-490e-8a73-aa24e189c41d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:55.766606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ce29b-bdcd-490e-8a73-aa24e189c41d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:55.766762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ce29b-bdcd-490e-8a73-aa24e189c41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:55.820922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:55.868651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fd2ce29b-bdcd-490e-8a73-aa24e189c41d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:55.869852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fd2ce29b-bdcd-490e-8a73-aa24e189c41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:56.026407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 701/1370] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:56.656487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:56.656925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:56.657408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:56.660214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 670/1371] 10.222.0.99 () {54 vars in 964 bytes} [Fri Sep 2 21:16:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:16:56.667341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56df9-f24e-4bdd-a2c6-ce676ddef0c9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:56.667821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56df9-f24e-4bdd-a2c6-ce676ddef0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:56.668267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c56df9-f24e-4bdd-a2c6-ce676ddef0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:56.803988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-19c56df9-f24e-4bdd-a2c6-ce676ddef0c9 None None] MFA Rules not processed for user `ba2b8b21f0454436a1958cb3abb27e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:56.806336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-19c56df9-f24e-4bdd-a2c6-ce676ddef0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:56.913957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:56.958370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 702/1372] 10.222.0.99 () {58 vars in 1061 bytes} [Fri Sep 2 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:16:56.984804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:56.994117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:57.110830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:57.148176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06a04-2788-457c-a11e-cf7fd3bbd3b6 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:57.148630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06a04-2788-457c-a11e-cf7fd3bbd3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:57.149067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e06a04-2788-457c-a11e-cf7fd3bbd3b6 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:57.152273 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 703/1373] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:16:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:16:57.159661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:57.160112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:57.160557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:57.204596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414e1de-bc6b-4aa6-a048-340cc3cbbdfe None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:57.250810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:16:57.252225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:57.307148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:57.307770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:57.308044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:57.308312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:57.310071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:57.381046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 704/1374] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:16:57.445370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:57.464184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:57.466000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913473c-cb42-4ed2-a717-51c2a2c0dcbd service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:57.492029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 671/1375] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:16:57.707648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:16:57.709961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:57.801848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:57.918799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc0936-39c7-4059-99df-cb6c8f741f34 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:16:58.088516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:16:58.089462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:16:58.089965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:16:58.090431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:16:58.094155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:16:58.176796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:16:58.249784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:16:58.251683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2365a7-2f49-42bc-b773-3ba0a464f04e service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:16:58.284780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 705/1376] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:00.605774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:00.606331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:00.606414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:00.607829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 672/1377] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:00.614047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdcddd-f345-483a-b3ea-a94d283e29aa None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:00.615981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdcddd-f345-483a-b3ea-a94d283e29aa None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:00.615981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdcddd-f345-483a-b3ea-a94d283e29aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:00.727010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-bcfdcddd-f345-483a-b3ea-a94d283e29aa None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:00.729062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bcfdcddd-f345-483a-b3ea-a94d283e29aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:00.902166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 706/1378] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:01.210932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:01.214071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:01.426836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd22b9-6156-4f7c-9231-7876f5e28e83 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:01.511812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:01.593907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:01.594923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:01.595390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:01.596087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:01.599304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:01.738733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:01.743649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09d753-439d-4e89-9e21-1addbf09ad1e service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:01.763722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 673/1379] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:04.492488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:04.494589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:04.494589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:04.496420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 707/1380] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:04.505324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6d82d-35f3-43f2-be90-c555e695807d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:04.506437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6d82d-35f3-43f2-be90-c555e695807d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:04.506846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6d82d-35f3-43f2-be90-c555e695807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:04.604017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a6b6d82d-35f3-43f2-be90-c555e695807d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:04.604017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a6b6d82d-35f3-43f2-be90-c555e695807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:04.630039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:04.797042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 674/1381] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:05.246046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:05.247889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:05.278865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:05.455351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ea581-aa42-42d6-8d4b-f596539be2c4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:05.604136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:05.628822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:05.629450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:05.629746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:05.630001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:05.631824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:05.797167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:05.800413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9026ce-a54f-40cc-98e2-6949404b52e0 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:05.830279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 708/1382] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:08.716868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:08.717743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:08.718392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:08.722103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 675/1383] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:08.730891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0090ca-c984-45d5-b4f4-d5657e3b2891 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:08.731762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0090ca-c984-45d5-b4f4-d5657e3b2891 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:08.731941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0090ca-c984-45d5-b4f4-d5657e3b2891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:08.854535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5b0090ca-c984-45d5-b4f4-d5657e3b2891 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:08.856050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5b0090ca-c984-45d5-b4f4-d5657e3b2891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:08.941591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:09.005598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 709/1384] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:17:09.323767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:09.326138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:09.588737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:09.592725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34123e4-2ae1-4066-a295-abf7d0ae51a8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:09.811535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:09.812909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:09.813418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:09.814004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:09.816396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:09.984720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:09.988786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03d570f-a742-440c-8419-9d4c594817d6 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:10.032827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 676/1385] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:12.891753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:12.892579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:12.892579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:12.894727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 710/1386] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:12.904336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50337c-60bd-4f2e-a842-421d14795b75 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:12.905157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50337c-60bd-4f2e-a842-421d14795b75 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:12.905336 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50337c-60bd-4f2e-a842-421d14795b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:12.913679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:13.041116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-da50337c-60bd-4f2e-a842-421d14795b75 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:13.043353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-da50337c-60bd-4f2e-a842-421d14795b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:13.235034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 677/1387] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:13.780121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:13.782261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:13.987302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69cfcb-3c7c-4705-8edd-60cfceb499d0 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:14.041457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:14.153694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:14.154335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:14.154607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:14.154893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:14.156711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:14.311738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:14.315589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587614cb-f635-4ec1-af52-55b6d72eee95 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:14.356062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 711/1388] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:15.437822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f7a-803c-4d6f-b677-253e93794d0a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:15.438385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f7a-803c-4d6f-b677-253e93794d0a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:15.438385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66f7a-803c-4d6f-b677-253e93794d0a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:15.439678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 678/1389] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:15.446590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2ff1-3547-49a3-9db7-e7ab19166898 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:15.447150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2ff1-3547-49a3-9db7-e7ab19166898 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:15.447150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2ff1-3547-49a3-9db7-e7ab19166898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:15.518225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:15.527113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-950b2ff1-3547-49a3-9db7-e7ab19166898 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:15.528507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-950b2ff1-3547-49a3-9db7-e7ab19166898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:15.648244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 712/1390] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:15.967183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:15.967869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:15.968186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:15.969849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 679/1391] 10.222.0.99 () {54 vars in 961 bytes} [Fri Sep 2 21:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:15.977570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674e09-a83d-4072-a968-7fd29e15e1f5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:15.978161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674e09-a83d-4072-a968-7fd29e15e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:15.978641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a674e09-a83d-4072-a968-7fd29e15e1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:16.105269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2a674e09-a83d-4072-a968-7fd29e15e1f5 None None] MFA Rules not processed for user `be3a2d2b02f44ce9bd64ffd38675b6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:16.107325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2a674e09-a83d-4072-a968-7fd29e15e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:16.222523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:16.233234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 713/1392] 10.222.0.99 () {58 vars in 1058 bytes} [Fri Sep 2 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:16.247186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:16.249328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:16.387280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:16.431401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfd139-78d0-41bd-a327-1aeb8931e7be None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:16.562673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:16.563712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:16.563987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:16.564382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:16.566610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:16.765841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:16.767735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e950d36-9323-4135-b41d-6ceeced45884 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:16.799136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 680/1393] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:17.228637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:17.229454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:17.229454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:17.230735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 714/1394] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:17.237322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecbc5-4fe4-4523-87cf-83cdd9faf3dc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:17.241390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecbc5-4fe4-4523-87cf-83cdd9faf3dc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:17.241390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ecbc5-4fe4-4523-87cf-83cdd9faf3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:17.256929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:17.324944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b21ecbc5-4fe4-4523-87cf-83cdd9faf3dc None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:17.326502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b21ecbc5-4fe4-4523-87cf-83cdd9faf3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:17.506534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 681/1395] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:17:17.879134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:17.880765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:17.984285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41c34-b241-45cc-9d82-43f7dffe233f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:17.989707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41c34-b241-45cc-9d82-43f7dffe233f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:17.989707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41c34-b241-45cc-9d82-43f7dffe233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:18.074434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-91f41c34-b241-45cc-9d82-43f7dffe233f None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:18.075653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-91f41c34-b241-45cc-9d82-43f7dffe233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:18.087036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6ae3f-5402-4bfd-bb61-8f61373e3a73 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:18.128161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:18.176998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:18.196233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 682/1396] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:17:18.241393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:18.242253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:18.242696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:18.243124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:18.246241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:18.268992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d7ee-e2cb-4d6c-96d3-28ba9672c5cb None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:18.269297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d7ee-e2cb-4d6c-96d3-28ba9672c5cb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:18.269567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d7ee-e2cb-4d6c-96d3-28ba9672c5cb None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:18.270995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 683/1397] 10.222.0.99 () {54 vars in 943 bytes} [Fri Sep 2 21:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:18.281350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11876c25-fdbb-4d12-9757-6f841f8c02f7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:18.281624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11876c25-fdbb-4d12-9757-6f841f8c02f7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:18.281930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11876c25-fdbb-4d12-9757-6f841f8c02f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:18.374690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-11876c25-fdbb-4d12-9757-6f841f8c02f7 None None] MFA Rules not processed for user `4e95d57ba23a424e8ce7e7e1695cd8a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:18.375928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-11876c25-fdbb-4d12-9757-6f841f8c02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:18.383705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:18.385398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:18.407062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 715/1398] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:18.512021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 684/1399] 10.222.0.99 () {58 vars in 1040 bytes} [Fri Sep 2 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:18.523119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:18.525125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:18.637368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:18.646689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c8f8f-7c9d-43cc-a4c3-36bde825765b service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:18.792536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:18.793364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:18.793644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:18.793954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:18.795850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:18.926359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:18.928359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69752a1-cc3c-4972-b421-a4f09ba6313e service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:18.940185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 716/1400] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:17:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 21:17:20.670556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:20.671381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:20.671381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:20.672442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 685/1401] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:17:20.678524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004d6b8-7163-46ef-a250-126fc0ccd8f8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:20.679095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004d6b8-7163-46ef-a250-126fc0ccd8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:20.679921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004d6b8-7163-46ef-a250-126fc0ccd8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:20.760501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:20.828698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8004d6b8-7163-46ef-a250-126fc0ccd8f8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:20.830727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8004d6b8-7163-46ef-a250-126fc0ccd8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:21.005781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 717/1402] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:21.415862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:21.419318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:21.446590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:21.690786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96155d4b-5d3a-480f-a4dc-167e537e9645 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:21.888619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:21.890000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:21.890355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:21.890696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:21.892467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:22.008300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:22.010066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817cb86-a57d-4fca-826c-ec27ace90e05 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:22.038509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 686/1403] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:22.224655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:24.646579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:24.647467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:24.647467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:24.649427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 718/1404] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:17:24.655890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f7a0f-323e-41f1-b155-8bd9502a7d1e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:24.656479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f7a0f-323e-41f1-b155-8bd9502a7d1e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:24.656893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f7a0f-323e-41f1-b155-8bd9502a7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:24.749044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-022f7a0f-323e-41f1-b155-8bd9502a7d1e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:24.750460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-022f7a0f-323e-41f1-b155-8bd9502a7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:24.847870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:24.929150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 687/1405] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:17:25.377441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:25.379813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:25.573299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:25.670979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bb653-daa7-4622-bd47-703694776839 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:25.812301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:25.816647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:25.816954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:25.817203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:25.819116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:25.908240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:25.946082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:25.963689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee960-b837-4774-800a-789ef25f8243 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:26.010497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 719/1406] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:28.665489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:28.666235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:28.666491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:28.673876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 688/1407] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:17:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:28.677742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560893-5778-4965-aeec-15d8093e7886 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:28.678593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560893-5778-4965-aeec-15d8093e7886 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:28.684842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560893-5778-4965-aeec-15d8093e7886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:28.801513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a2560893-5778-4965-aeec-15d8093e7886 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:28.803037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a2560893-5778-4965-aeec-15d8093e7886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:28.942600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 720/1408] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:29.254870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:29.258173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:29.573576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce62d4-1089-483f-be84-2080257488d7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:29.655705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:29.785166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:29.786248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:29.786751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:29.787194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:29.790631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:29.950777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:29.952756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4e10-6dd0-4e79-ab51-13e4deb0f79c service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:29.979193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 689/1409] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:17:38.940530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=80 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:39.975776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=83 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:45.050831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f3cb-1505-4e25-81f7-98b7b4a0839c None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:45.051408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f3cb-1505-4e25-81f7-98b7b4a0839c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:45.051408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f3cb-1505-4e25-81f7-98b7b4a0839c None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:45.052756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 721/1410] 10.222.0.99 () {54 vars in 918 bytes} [Fri Sep 2 21:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:45.058438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5a5-d7ca-4846-822c-11079011b6c0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:45.058947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5a5-d7ca-4846-822c-11079011b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:45.059060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094f5a5-d7ca-4846-822c-11079011b6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:45.148643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b094f5a5-d7ca-4846-822c-11079011b6c0 None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:45.149990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b094f5a5-d7ca-4846-822c-11079011b6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:45.269017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 690/1411] 10.222.0.99 () {58 vars in 1015 bytes} [Fri Sep 2 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:45.286023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c4029-b39b-4aaa-8f1f-83d3afccb682 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:45.286290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c4029-b39b-4aaa-8f1f-83d3afccb682 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:45.286564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c4029-b39b-4aaa-8f1f-83d3afccb682 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:45.288137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 722/1412] 10.222.0.99 () {54 vars in 918 bytes} [Fri Sep 2 21:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:17:45.294003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8266-8c07-4bd4-9465-52c0f877d6d7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:45.294312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8266-8c07-4bd4-9465-52c0f877d6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:45.294526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee8266-8c07-4bd4-9465-52c0f877d6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:45.390195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:45.393147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f9ee8266-8c07-4bd4-9465-52c0f877d6d7 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:45.394431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f9ee8266-8c07-4bd4-9465-52c0f877d6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:45.543522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 691/1413] 10.222.0.99 () {58 vars in 1015 bytes} [Fri Sep 2 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:17:48.046477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:48.047073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:48.047073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:48.048372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 723/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:17:48.054049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86776-5a34-4e22-aead-c9753b17547f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:48.054556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86776-5a34-4e22-aead-c9753b17547f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:48.054659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86776-5a34-4e22-aead-c9753b17547f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:48.137357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fdf86776-5a34-4e22-aead-c9753b17547f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:48.138624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fdf86776-5a34-4e22-aead-c9753b17547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:48.201612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:48.250907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 692/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:48.546491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:48.548091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:48.730097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adae8e2-f046-46a8-960b-97fbf2dd0fb0 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:48.832566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:48.887646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:48.888267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:48.888541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:48.888812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:48.890517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:49.001432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:49.003029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ca05c-436f-45d1-a40a-ff06a831cfd6 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:49.026066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 724/1416] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:50.026845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:50.027440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:50.027440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:50.028866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 693/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:50.033915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593992a-7362-4c48-872f-8dcd00e20f88 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:50.034442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593992a-7362-4c48-872f-8dcd00e20f88 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:50.034442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593992a-7362-4c48-872f-8dcd00e20f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:50.125482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f593992a-7362-4c48-872f-8dcd00e20f88 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:50.126743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f593992a-7362-4c48-872f-8dcd00e20f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:50.156366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:50.240049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 725/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:50.399344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:50.401072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:50.566906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479076b-2253-413b-b1c4-0ed6f96db4b0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:50.613335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:50.698215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:50.698795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:50.699349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:50.699621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:50.701254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:50.801412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:50.803125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7475-b5b4-4aa2-af45-0f499c75ac00 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:50.824773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 694/1419] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:51.756165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:51.756904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:51.756904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:51.758085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 726/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:51.763608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a470783-dd67-4997-904a-25bce54bf2ff None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:51.764641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a470783-dd67-4997-904a-25bce54bf2ff None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:51.764641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a470783-dd67-4997-904a-25bce54bf2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:51.823755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:51.846730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8a470783-dd67-4997-904a-25bce54bf2ff None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:51.847949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a470783-dd67-4997-904a-25bce54bf2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:51.957084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 695/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:52.108859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:52.110462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:52.274303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca857e9-bbef-4206-8ee5-2bfb331c2e8a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:52.287376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:52.400193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:52.400810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:52.401082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:52.401346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:52.403079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:52.498624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:52.500311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b3c12-5bde-4f74-9d11-3794d719ce3b service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:52.520266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 727/1422] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:53.430586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:53.431153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:53.431153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:53.432357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 696/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:53.438173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c860c7-8b5e-4e92-a3cf-6fdbc2d78e9e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:53.439037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c860c7-8b5e-4e92-a3cf-6fdbc2d78e9e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:53.439037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c860c7-8b5e-4e92-a3cf-6fdbc2d78e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:53.459619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:53.526950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f1c860c7-8b5e-4e92-a3cf-6fdbc2d78e9e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:53.528191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f1c860c7-8b5e-4e92-a3cf-6fdbc2d78e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:53.635282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 728/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:53.789315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:53.791816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:53.907743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:53.947889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867c326-1355-441e-9920-7bcdef7b593b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:54.078447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:54.078964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:54.079241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:54.079511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:54.081139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:54.172504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:54.186134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:54.187739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8cce8-00a6-4092-a411-5752c73850c8 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:54.206190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 697/1425] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:55.104481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:55.105076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:55.105076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:55.106416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 729/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:55.111763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e25da7-556d-4081-8278-309677de48c8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:55.112481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e25da7-556d-4081-8278-309677de48c8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:55.112481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e25da7-556d-4081-8278-309677de48c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:55.200842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a4e25da7-556d-4081-8278-309677de48c8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:55.202166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a4e25da7-556d-4081-8278-309677de48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:55.307417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 698/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:55.459198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:55.461106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:55.560137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:55.630632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afa281-29ec-486c-aa35-7528865f70c9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:55.748864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:55.749474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:55.749769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:55.750039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:55.751719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:55.801998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:55.843553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:55.845066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afe5f4-be2f-49c9-8097-1fe95fa3a320 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:55.864771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 730/1428] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:56.833157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:56.834482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:56.834482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:56.835762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 699/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:56.841161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae5f9d-a00c-45ff-87dd-957b699715b8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:56.841652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae5f9d-a00c-45ff-87dd-957b699715b8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:56.841794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae5f9d-a00c-45ff-87dd-957b699715b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:56.923877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-60ae5f9d-a00c-45ff-87dd-957b699715b8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:56.925028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-60ae5f9d-a00c-45ff-87dd-957b699715b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:57.025643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 731/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:57.192135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:57.194895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:57.247905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:57.346580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64feeb-6bb0-4057-a132-d9d61c25b52c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:57.473091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:57.473703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:57.474588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:57.474947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:57.476707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:57.485089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:57.570744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:57.572364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e5254-0927-4b5a-b02a-c37ac2740a52 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:57.590838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 700/1431] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:17:58.470310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:58.470798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:58.470925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:58.472237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 732/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:17:58.477903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954ac8-588c-479a-b2c2-ba3b7df0d69c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:58.478407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954ac8-588c-479a-b2c2-ba3b7df0d69c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:58.478523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa954ac8-588c-479a-b2c2-ba3b7df0d69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:58.570422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-aa954ac8-588c-479a-b2c2-ba3b7df0d69c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:17:58.571657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-aa954ac8-588c-479a-b2c2-ba3b7df0d69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:58.763731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:58.795201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 701/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:17:58.959327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:17:58.960850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:58.979096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:59.110597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1237beb-f9c4-40bc-aeb5-7a9db522fb67 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:17:59.227991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:17:59.375895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:17:59.376394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:17:59.376668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:17:59.376928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:17:59.378531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:17:59.468508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:17:59.470129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e89573-9e95-465e-8eda-880e36e69df6 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:17:59.489427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 733/1434] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:00.390226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:00.390805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:00.390805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:00.392133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 702/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:00.397598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb78ad-b82d-4ec2-be96-1740891c35a2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:00.397912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb78ad-b82d-4ec2-be96-1740891c35a2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:00.398185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb78ad-b82d-4ec2-be96-1740891c35a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:00.480910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6ffb78ad-b82d-4ec2-be96-1740891c35a2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:00.482194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6ffb78ad-b82d-4ec2-be96-1740891c35a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:00.529029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:00.594528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 734/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:18:00.754673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:00.756147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:00.917150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6ecef-d9cb-4a6d-bdae-76b878b86bf7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:00.976195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:01.038489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:01.039103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:01.039373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:01.039636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:01.041227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:01.136648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:01.137561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ca561-0677-4d53-a8dc-3c23b06d4179 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:01.155804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 703/1437] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:02.050208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:02.050786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:02.050786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:02.052786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 735/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:18:02.058220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e1b6-78a3-463c-b1d8-18012b27a3b6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:02.058689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e1b6-78a3-463c-b1d8-18012b27a3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:02.058794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e1b6-78a3-463c-b1d8-18012b27a3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:02.140817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3a02e1b6-78a3-463c-b1d8-18012b27a3b6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:02.142036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3a02e1b6-78a3-463c-b1d8-18012b27a3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:02.144118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:02.244828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 704/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:02.399035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:02.400556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:02.549814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ff4a1-acca-4125-8e4d-aef4767c0241 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:02.578023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:02.671762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:02.672393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:02.672667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:02.672930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:02.674588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:02.775833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:02.777433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e3c57-80b8-46ce-bc80-e056158d3ec4 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:02.796884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 736/1440] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:04.705571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:04.706157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:04.706157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:04.707477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 705/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:04.713604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51172e-c3cb-4f99-ba99-403961ffbf32 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:04.714253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51172e-c3cb-4f99-ba99-403961ffbf32 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:04.714253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51172e-c3cb-4f99-ba99-403961ffbf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:04.760906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:04.787487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ec51172e-c3cb-4f99-ba99-403961ffbf32 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:04.788687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ec51172e-c3cb-4f99-ba99-403961ffbf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:04.903257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 737/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:18:05.199021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:05.200541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:05.314592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235b080-b1f0-4648-b89c-a0d27b0f8838 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:05.358999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:05.404843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:05.405448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:05.405748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:05.406020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:05.407609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:05.528822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:05.530354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912d81e-3526-4c23-ad02-e70968c34b3f service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:05.550237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 706/1443] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:07.662599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f12a-66b6-458f-af16-107727ab193f None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:07.663240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f12a-66b6-458f-af16-107727ab193f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:07.663240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f12a-66b6-458f-af16-107727ab193f None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:07.664577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 738/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:07.670295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd0a57-33bb-471d-9c60-2388a8fd33eb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:07.670719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd0a57-33bb-471d-9c60-2388a8fd33eb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:07.670851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd0a57-33bb-471d-9c60-2388a8fd33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:07.744398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-58fd0a57-33bb-471d-9c60-2388a8fd33eb None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:07.749344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-58fd0a57-33bb-471d-9c60-2388a8fd33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:07.759909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:07.853798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 707/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:08.152791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f12a-66b6-458f-af16-107727ab193f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:08.154402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4922f12a-66b6-458f-af16-107727ab193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:08.264428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f12a-66b6-458f-af16-107727ab193f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:08.277029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:08.343399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:08.343920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:08.344303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:08.344575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:08.346278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:08.471609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:08.473035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3173b9-b214-400d-a6a0-779b1172e6b6 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:08.493105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 739/1446] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:11.071921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:11.072514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:11.072514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:11.073806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 708/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:11.079631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3559e3-f36c-4585-afe5-eb675739d4ce None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:11.080160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3559e3-f36c-4585-afe5-eb675739d4ce None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:11.080160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3559e3-f36c-4585-afe5-eb675739d4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:11.131091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:11.148741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fc3559e3-f36c-4585-afe5-eb675739d4ce None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:11.149986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fc3559e3-f36c-4585-afe5-eb675739d4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:11.249110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 740/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:11.570392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:11.571910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:11.681371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe2a6b-0e00-4301-9439-2a001c70f085 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:11.722373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:11.759129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:11.759737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:11.760005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:11.760268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:11.761939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:11.874761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:11.876145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75186bc6-c8c6-494d-a329-760ce0209c61 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:11.894636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 709/1449] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:14.041146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:14.041707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:14.041707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:14.043098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 741/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:14.048965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591b2d6-b23f-4f44-857c-fedeef37ea91 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:14.049484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591b2d6-b23f-4f44-857c-fedeef37ea91 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:14.049484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591b2d6-b23f-4f44-857c-fedeef37ea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:14.123825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4591b2d6-b23f-4f44-857c-fedeef37ea91 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:14.125048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4591b2d6-b23f-4f44-857c-fedeef37ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:14.146189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:14.230754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 710/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:14.533330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:14.535007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:14.653496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a90bc0-be3a-4eb2-a8fa-4bb9f3f74d42 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:14.674405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:14.730358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:14.731122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:14.731437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:14.731668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:14.733324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:14.854091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:14.855472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef08be-14b7-48a7-9bd4-99e69777f292 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:14.875497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 742/1452] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:16.911043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:16.911581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:16.911581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:16.912923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 711/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:16.918589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a8bf2-2da0-4633-97d6-5a93e353e562 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:16.919041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a8bf2-2da0-4633-97d6-5a93e353e562 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:16.919146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a8bf2-2da0-4633-97d6-5a93e353e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:16.986457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:16.990583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cb7a8bf2-2da0-4633-97d6-5a93e353e562 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:16.992137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cb7a8bf2-2da0-4633-97d6-5a93e353e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:17.094237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 743/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:17.386792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:17.388337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:17.511874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3137b8b-bdf3-46e2-8116-6646ffe9707d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:17.564348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:17.589633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:17.590277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:17.590564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:17.590822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:17.592521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:17.709847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:17.711194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769c1e6-0594-495a-b277-ea73ae1330e7 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:17.730108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 712/1455] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:19.801789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:19.802563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:19.802563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:19.803705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 744/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:19.809547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f022c-eeeb-4dfc-82b8-e8e75c100556 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:19.810117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f022c-eeeb-4dfc-82b8-e8e75c100556 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:19.810117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f022c-eeeb-4dfc-82b8-e8e75c100556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:19.889802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-033f022c-eeeb-4dfc-82b8-e8e75c100556 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:19.890948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-033f022c-eeeb-4dfc-82b8-e8e75c100556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:19.923175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:20.001104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 713/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:20.305383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:20.306981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:20.415805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd7f38-5953-48d1-ab00-7c527f261deb None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:20.445302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:20.497061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:20.497667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:20.497976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:20.498304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:20.499903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:20.622314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:20.623793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda607c-8954-4b46-a958-5f9f740563e0 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:20.643065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 745/1458] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:22.726495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:22.727321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:22.727486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:22.728903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 714/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:18:22.735221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47738b7-f172-4cb5-9082-04b9ad9e7c53 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:22.735745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47738b7-f172-4cb5-9082-04b9ad9e7c53 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:22.735745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47738b7-f172-4cb5-9082-04b9ad9e7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:22.812606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b47738b7-f172-4cb5-9082-04b9ad9e7c53 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:22.813942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b47738b7-f172-4cb5-9082-04b9ad9e7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:22.823327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:22.939516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 746/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:23.233682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:23.235308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:23.356107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da68b8-58f3-469e-b96d-de515ba775d4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:23.423364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:23.445095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:23.447227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:23.447512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:23.448953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:23.450732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:23.607633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:23.609048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b433284-412b-447b-b627-36caa0f6a211 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:23.632237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 715/1461] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:25.740693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:25.741235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:25.741235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:25.742932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 747/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:25.749508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975066-4796-4a32-8e0e-dd70fdc404cd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:25.750150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975066-4796-4a32-8e0e-dd70fdc404cd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:25.750150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975066-4796-4a32-8e0e-dd70fdc404cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:25.832464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-49975066-4796-4a32-8e0e-dd70fdc404cd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:25.833745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-49975066-4796-4a32-8e0e-dd70fdc404cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:25.878026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:25.947913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 716/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:26.241264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:26.243158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:26.375494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660eba0-2cc1-47aa-bc6b-1200780a483e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:26.415390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:26.455438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:26.456055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:26.456615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:26.457350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:26.459073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:26.584746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:26.586250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a970974-12b8-477b-bc05-64e06e9c2e4b service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:26.610614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 748/1464] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:28.632806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:28.633387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:28.633387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:28.634702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 717/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:28.640536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234404-9d3e-4acf-9f6f-8417c954d612 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:28.641112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234404-9d3e-4acf-9f6f-8417c954d612 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:28.641112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234404-9d3e-4acf-9f6f-8417c954d612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:28.712234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-77234404-9d3e-4acf-9f6f-8417c954d612 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:28.713462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-77234404-9d3e-4acf-9f6f-8417c954d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:28.733931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:28.819168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 749/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:29.115965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:29.117511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:29.248454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a4dd7-4757-4ef1-afab-239476e7dc44 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:29.323386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:29.339028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:29.341213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:29.341213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:29.341213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:29.344234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:29.477768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:29.479225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e890ee-de58-4e20-9097-7dbee4408e82 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:29.499413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 718/1467] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:31.549552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:31.550167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:31.550301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:31.551756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 750/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:31.557422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201ed10-ab63-4f5b-8203-d98a5bc2c522 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:31.557914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201ed10-ab63-4f5b-8203-d98a5bc2c522 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:31.558020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201ed10-ab63-4f5b-8203-d98a5bc2c522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:31.636001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b201ed10-ab63-4f5b-8203-d98a5bc2c522 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:31.637230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b201ed10-ab63-4f5b-8203-d98a5bc2c522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:31.687348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:31.739625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 719/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:32.025919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:32.027634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:32.152073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5b85e-7f2f-4a91-bdcb-f153597d0a43 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:32.214294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:32.250544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:32.251305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:32.251611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:32.251875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:32.253910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:32.378285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:32.380136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fd9d1-c87d-4ab9-9067-8565068250f8 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:32.401663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 751/1470] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:41.737126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=22 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:42.399305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=66 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:52.996599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2c1-dad5-446c-a57f-72695767d6ee None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:52.997208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2c1-dad5-446c-a57f-72695767d6ee None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:52.997208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a2c1-dad5-446c-a57f-72695767d6ee None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:52.998520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 720/1471] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:53.004993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31d6f-8dc1-4051-801a-016d0662f341 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:53.005555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31d6f-8dc1-4051-801a-016d0662f341 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:53.005702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b31d6f-8dc1-4051-801a-016d0662f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:53.087018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-33b31d6f-8dc1-4051-801a-016d0662f341 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:53.088341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-33b31d6f-8dc1-4051-801a-016d0662f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:53.206104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 752/1472] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:53.515796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:53.516075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:53.516342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:53.518936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 721/1473] 10.222.0.99 () {54 vars in 961 bytes} [Fri Sep 2 21:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:53.523692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805cb7-c3d2-4878-9f54-3d3e53c09715 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:53.523979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805cb7-c3d2-4878-9f54-3d3e53c09715 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:53.524282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24805cb7-c3d2-4878-9f54-3d3e53c09715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:53.607350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-24805cb7-c3d2-4878-9f54-3d3e53c09715 None None] MFA Rules not processed for user `be3a2d2b02f44ce9bd64ffd38675b6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:53.608567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-24805cb7-c3d2-4878-9f54-3d3e53c09715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:53.631849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:53.694181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 753/1474] 10.222.0.99 () {58 vars in 1058 bytes} [Fri Sep 2 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:53.701828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:53.703543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:53.816779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2dd54-21aa-4ffd-bb4e-3cafa3363082 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:53.903914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:53.904518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:53.904793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:53.905059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:53.906717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:53.965806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:54.023728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:54.025164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a413203-d2c8-4c2d-b87e-d15e8b5f0ce0 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:54.044412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 722/1475] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:18:56.144778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:56.145344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:56.145344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:56.146735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 754/1476] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:56.152540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a8b97-9db3-46d4-a396-b6756c85aaec None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:56.153287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a8b97-9db3-46d4-a396-b6756c85aaec None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:56.153287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a8b97-9db3-46d4-a396-b6756c85aaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:56.222826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-110a8b97-9db3-46d4-a396-b6756c85aaec None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:56.224050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-110a8b97-9db3-46d4-a396-b6756c85aaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:56.323424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 723/1477] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:18:56.620280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:56.621859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:56.761121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e0bd2-c1f6-46e6-83e0-2f6903edc045 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:56.813453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:56.860685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:56.862153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:56.862437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:56.862696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:56.863318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:56.865102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:18:56.865102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:18:56.865102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:18:56.866288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9092a0b-2db1-40ce-822f-05616ba5a0de admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:56.886125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 755/1478] 10.222.0.99 () {56 vars in 1108 bytes} [Fri Sep 2 21:18:56 2022] GET /identity/v3/projects => generated 2553 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:18:58.864704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9727d-b121-4483-a198-20915c50049c None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:58.865470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9727d-b121-4483-a198-20915c50049c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:58.865470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9727d-b121-4483-a198-20915c50049c None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:58.866508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 724/1479] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:18:58.871895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d267cd-0d6d-4936-870a-db2b12177ae2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:58.872317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d267cd-0d6d-4936-870a-db2b12177ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:58.872421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d267cd-0d6d-4936-870a-db2b12177ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:58.942874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-04d267cd-0d6d-4936-870a-db2b12177ae2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:18:58.944099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-04d267cd-0d6d-4936-870a-db2b12177ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:59.042760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 756/1480] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:18:59.342686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9727d-b121-4483-a198-20915c50049c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:18:59.344286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b3b9727d-b121-4483-a198-20915c50049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:59.357669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:59.501349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9727d-b121-4483-a198-20915c50049c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:18:59.585332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:18:59.619319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:18:59.619949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:18:59.620198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:18:59.620522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:18:59.622080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:18:59.725896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:18:59.727592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be16e8-c92f-434d-9f39-b2e4e7c4bafa service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:18:59.747366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 725/1481] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:19:01.937542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:01.938231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:01.938231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:01.940151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 757/1482] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:19:01.946688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4d73-c482-4e54-9257-3382e4e127bd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:01.947200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4d73-c482-4e54-9257-3382e4e127bd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:01.947306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb4d73-c482-4e54-9257-3382e4e127bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:02.018314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-16bb4d73-c482-4e54-9257-3382e4e127bd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:19:02.019533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-16bb4d73-c482-4e54-9257-3382e4e127bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:02.072457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:02.126324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 726/1483] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:19:02.420495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:19:02.422856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:02.450759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:02.546867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0c85f-2a8f-4d6e-9c84-a0de5b532e9a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:19:02.624796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:19:02.625397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:02.625667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:02.625967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:02.627798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:02.713944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:02.750149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:19:02.752986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77716-3334-4d31-9759-5db01f0747d9 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:19:02.773232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 758/1484] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:19:04.827114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:04.827681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:04.827681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:04.828999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 727/1485] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:19:04.834734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e20c6-9bb4-4c0b-a0f2-10e02a6aa0f6 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:04.835540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e20c6-9bb4-4c0b-a0f2-10e02a6aa0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:04.835717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e20c6-9bb4-4c0b-a0f2-10e02a6aa0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:04.905753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5b7e20c6-9bb4-4c0b-a0f2-10e02a6aa0f6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:19:04.906897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5b7e20c6-9bb4-4c0b-a0f2-10e02a6aa0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:05.013835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 759/1486] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:19:05.309664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:19:05.311298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:05.427726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abdefa-b66c-41b1-a564-109e61762643 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:19:05.510896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:05.512839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:19:05.513472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:05.513774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:05.514045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:05.517508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:05.668673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:19:05.670328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a06c85-2b20-495f-8776-7fc0b79b8a29 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:19:05.690149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 728/1487] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:19:07.848071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:07.848649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:07.848649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:07.850028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 760/1488] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:19:07.856342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc86-4909-4cfc-82bd-6f1ee1b1b790 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:07.856921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc86-4909-4cfc-82bd-6f1ee1b1b790 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:07.856921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164fc86-4909-4cfc-82bd-6f1ee1b1b790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:07.936463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5164fc86-4909-4cfc-82bd-6f1ee1b1b790 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:19:07.937676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5164fc86-4909-4cfc-82bd-6f1ee1b1b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:08.004464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:08.054700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 729/1489] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:19:08.355548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:19:08.357117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:08.394111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:08.463632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b147ef7-e455-4364-a7c3-3551961767e9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:19:08.541747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:19:08.542419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:08.542694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:08.542961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:08.544586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:08.632533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:08.666347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:19:08.667957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150360-3552-4883-a309-4d73a3fd7d69 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:19:08.687527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 761/1490] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:19:11.086130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:11.086730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:11.086730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:11.088033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 730/1491] 10.222.0.99 () {54 vars in 927 bytes} [Fri Sep 2 21:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:19:11.093470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a18ee6-2a95-4163-bd8f-dd846998b4e2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:11.093997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a18ee6-2a95-4163-bd8f-dd846998b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:11.094114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a18ee6-2a95-4163-bd8f-dd846998b4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:11.164409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a3a18ee6-2a95-4163-bd8f-dd846998b4e2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:19:11.165693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a3a18ee6-2a95-4163-bd8f-dd846998b4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:11.279866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 762/1492] 10.222.0.99 () {58 vars in 1024 bytes} [Fri Sep 2 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:19:11.573574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:19:11.575590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:11.724396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d34ad2-5e49-41a0-973d-36e1a1625d6e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:19:11.809249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:19:11.809908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:19:11.810187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:19:11.810457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:19:11.812286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:19:11.830350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:11.936280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:19:11.937740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd9fdf-dc5a-406e-be45-ce2f64927a0d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:19:11.959101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 731/1493] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:19:21.278022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=82 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:19:21.956569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=50 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:20:39.328847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:39.329534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:39.329534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:39.444772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:20:39.446267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:39.623195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 763/1494] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:20:39.634030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:39.634659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:39.634659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:39.663901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:20:39.665126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:39.674619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 732/1495] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:20:39.679511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:39.692561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa008b2-52f4-427b-aa3f-be7073541052 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:39.697939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:39.698492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:39.698762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:39.699088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:39.700836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:20:39.701235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:39.702361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:39.702361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:20:39.702361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:20:39.702361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:39.721568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 764/1496] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:20:39.734609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:39.750105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1a4b4-e505-408f-8675-20f5c2bce597 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:39.754950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:39.755702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:39.755979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:39.756249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:39.756984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:39.759048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:39.759048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:20:39.759048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:20:39.760396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:39.953073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 733/1497] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:20:39 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:20:39.957895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:39.971637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622eb49d-db8f-4b8e-8a77-c67015503c21 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.038941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:20:40.047060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:20:40.051840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:20:40.054231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:40.243024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 765/1498] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:20:39 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:20:40.247636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:40.258246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8c6e-144b-4569-a3e9-b1f77a53c2fd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.263937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:20:40.266855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:20:40.267587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:40.281681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 734/1499] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:20:40.285798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:40.296701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a30e5b-5391-4676-8f23-6ccf98c4cc0f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.301895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.302510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.302796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.303060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.303706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.304728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:40.304728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:20:40.304728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:20:40.305624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a0268-46c9-4b83-821c-971210f82de5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:40.320428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 766/1500] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:20:40.325067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:40.335300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11924341-029f-4f02-927a-00bd60eb1ef0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.340565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.341206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.341481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.341783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] PATH_INFO: `/v3/projects/3686ba2d82114a78ab585be83314c27f/users/abd7bbb7032e49e09ab0dcedd58e6a4c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.357350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3686ba2d82114a78ab585be83314c27f, user_id=abd7bbb7032e49e09ab0dcedd58e6a4c, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.358678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:40.358678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:20:40.358678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:20:40.360610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724235c-127e-4658-bb16-75563d2a278c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:40.441814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 735/1501] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:20:40 2022] PUT /identity/v3/projects/3686ba2d82114a78ab585be83314c27f/users/abd7bbb7032e49e09ab0dcedd58e6a4c/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:20:40.454817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cf22-4f12-45d5-87b0-0b8ba9e32c9f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.455361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cf22-4f12-45d5-87b0-0b8ba9e32c9f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.455361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450cf22-4f12-45d5-87b0-0b8ba9e32c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.484845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6450cf22-4f12-45d5-87b0-0b8ba9e32c9f None None] MFA Rules not processed for user `abd7bbb7032e49e09ab0dcedd58e6a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:20:40.486076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6450cf22-4f12-45d5-87b0-0b8ba9e32c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:40.556127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 767/1502] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:20:40.569697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ca2a-c166-4dab-9433-11e4baebc868 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.570004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ca2a-c166-4dab-9433-11e4baebc868 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.570300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21ca2a-c166-4dab-9433-11e4baebc868 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.571629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 736/1503] 10.222.0.99 () {54 vars in 964 bytes} [Fri Sep 2 21:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 21:20:40.578001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb97dd-50a2-445e-95df-46a237849b7f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.578288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb97dd-50a2-445e-95df-46a237849b7f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.578567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb97dd-50a2-445e-95df-46a237849b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.643703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3cdb97dd-50a2-445e-95df-46a237849b7f None None] MFA Rules not processed for user `ba2b8b21f0454436a1958cb3abb27e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:20:40.644899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3cdb97dd-50a2-445e-95df-46a237849b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:40.726801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 768/1504] 10.222.0.99 () {58 vars in 1061 bytes} [Fri Sep 2 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:20:40.734817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21ca2a-c166-4dab-9433-11e4baebc868 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:40.745487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21ca2a-c166-4dab-9433-11e4baebc868 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.749251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.749888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.750152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.750416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.758763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.760057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:40.769542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 737/1505] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:20:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:20:40.817346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaded01-1cdd-417d-9573-0ec8fd929c02 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.817627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaded01-1cdd-417d-9573-0ec8fd929c02 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.817923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaded01-1cdd-417d-9573-0ec8fd929c02 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.819187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 769/1506] 10.222.0.99 () {52 vars in 830 bytes} [Fri Sep 2 21:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:20:40.822950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:40.888113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbb31c-f842-4548-9290-1a64a9590346 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:40.892798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:40.893444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:40.893733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:40.894011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] PATH_INFO: `/v3/users/abd7bbb7032e49e09ab0dcedd58e6a4c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:40.895160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7bbb7032e49e09ab0dcedd58e6a4c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:40.896173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:40.896173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:20:40.896173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:20:40.897404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:41.156041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7bbb7032e49e09ab0dcedd58e6a4c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:41.156705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7bbb7032e49e09ab0dcedd58e6a4c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:41.252837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7bbb7032e49e09ab0dcedd58e6a4c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:41.270052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7bbb7032e49e09ab0dcedd58e6a4c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:41.297317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:20:41.302815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7bbb7032e49e09ab0dcedd58e6a4c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:41.311457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 738/1507] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:20:40 2022] DELETE /identity/v3/users/abd7bbb7032e49e09ab0dcedd58e6a4c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:20:41.316250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:41.316508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:41.316772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:41.338222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:20:41.339479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:41.349535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:20:41.403274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 770/1508] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:20:41.414381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:41.415883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:41.500905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641807a-ded8-4056-998e-969e3a80639a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:41.504824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:41.505433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:41.505704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:41.506081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:41.516300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:41.517759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb0030-b75a-4e89-ad15-dfd7c71d3ba3 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:41.527805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 739/1509] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:20:42.100418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:20:42.101891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:20:42.114934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98330b5e-72f1-4854-99be-ae50fb4a9112 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:20:42.119497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:20:42.120130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:20:42.120401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:20:42.121553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] PATH_INFO: `/v3/projects/3686ba2d82114a78ab585be83314c27f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:20:42.122999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3686ba2d82114a78ab585be83314c27f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:20:42.124582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:20:42.124582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:20:42.124582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:20:42.125274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:20:42.287497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3686ba2d82114a78ab585be83314c27f', 'request_id': 'req-40a1714e-0dd4-4104-a5a9-8c31b93906f8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:42.301378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3686ba2d82114a78ab585be83314c27f', 'request_id': 'req-40a1714e-0dd4-4104-a5a9-8c31b93906f8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:42.389482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3686ba2d82114a78ab585be83314c27f', 'request_id': 'req-40a1714e-0dd4-4104-a5a9-8c31b93906f8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:42.401379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] The token cache is being invalidate because project 3686ba2d82114a78ab585be83314c27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:20:42.401660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:42.402510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-40a1714e-0dd4-4104-a5a9-8c31b93906f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:20:42.403991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 771/1510] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:20:42 2022] DELETE /identity/v3/projects/3686ba2d82114a78ab585be83314c27f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:20:51.523338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:20:52.398281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:22:17.955319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:17.955886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:17.955886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:17.957180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 740/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:17.962960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1b73-fc77-470d-ae2e-2d75d7d84f70 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:17.963428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1b73-fc77-470d-ae2e-2d75d7d84f70 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:17.963532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1b73-fc77-470d-ae2e-2d75d7d84f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:17.983676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b40a1b73-fc77-470d-ae2e-2d75d7d84f70 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:17.984883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b40a1b73-fc77-470d-ae2e-2d75d7d84f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:18.045522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 772/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:18.208360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:18.210189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:18.325614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdedc51-971c-440d-aa43-709a5e702f59 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:18.331344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:18.331948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:18.332225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:18.332494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:18.341830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:18.343556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c57c8-0295-47ed-94a8-241f4c4d2bed service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:18.353424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 741/1513] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:19.215626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b481-c8b7-4868-a72e-0e0d20464fbc None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:19.216440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b481-c8b7-4868-a72e-0e0d20464fbc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:19.216440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596b481-c8b7-4868-a72e-0e0d20464fbc None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:19.217526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 773/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:19.222756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20604-5d2b-435e-8624-04b628f97029 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:19.223160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20604-5d2b-435e-8624-04b628f97029 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:19.223299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac20604-5d2b-435e-8624-04b628f97029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:19.244452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1ac20604-5d2b-435e-8624-04b628f97029 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:19.245705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1ac20604-5d2b-435e-8624-04b628f97029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:19.253518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 742/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:19.408245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b481-c8b7-4868-a72e-0e0d20464fbc None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:19.423158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596b481-c8b7-4868-a72e-0e0d20464fbc None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:19.428528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:19.429185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:19.429435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:19.429708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:19.437916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:19.439505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10662-3b5a-4c0c-b9a3-11bb5c152397 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:19.449178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 774/1516] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:20.207732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93c098-af2f-46a2-b3d7-60e3efe0d650 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:20.208020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93c098-af2f-46a2-b3d7-60e3efe0d650 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:20.208299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93c098-af2f-46a2-b3d7-60e3efe0d650 None None] PATH_INFO: `/v3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:20.209669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 743/1517] 10.222.0.26 () {54 vars in 925 bytes} [Fri Sep 2 21:22:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:22:20.217868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0349-6804-4125-a707-7dc05063161d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:20.218419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0349-6804-4125-a707-7dc05063161d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:20.218419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0349-6804-4125-a707-7dc05063161d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:20.269791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-869a0349-6804-4125-a707-7dc05063161d None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:20.271012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-869a0349-6804-4125-a707-7dc05063161d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:20.343554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 775/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Sep 2 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:29.238927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=49 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:22:30.340633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=77 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:22:30.379027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe998-5564-45cb-a036-18e4705c72a7 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.379583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe998-5564-45cb-a036-18e4705c72a7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.379583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe998-5564-45cb-a036-18e4705c72a7 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.380889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 744/1519] 10.222.0.99 () {54 vars in 961 bytes} [Fri Sep 2 21:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:30.386931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c1791-605c-4d70-bf31-8d55b78fc032 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.387520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c1791-605c-4d70-bf31-8d55b78fc032 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.388120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c1791-605c-4d70-bf31-8d55b78fc032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.408953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0c1c1791-605c-4d70-bf31-8d55b78fc032 None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:30.410201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0c1c1791-605c-4d70-bf31-8d55b78fc032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:30.419302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 776/1520] 10.222.0.99 () {58 vars in 1058 bytes} [Fri Sep 2 21:22:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:22:30.427104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbe998-5564-45cb-a036-18e4705c72a7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:30.441313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbe998-5564-45cb-a036-18e4705c72a7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:30.447582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:30.448353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.448626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.449013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.457976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:30.459646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abb5fb-32d6-4383-b74f-e17c2269f112 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:30.469830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 745/1521] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:30.690339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5ece-5ceb-42d1-a4ce-e3dc08bc5144 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.690621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5ece-5ceb-42d1-a4ce-e3dc08bc5144 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.690926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5ece-5ceb-42d1-a4ce-e3dc08bc5144 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.692314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 777/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:30.698648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46cd4f-8071-468e-a723-66dd43d02ab9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.699060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46cd4f-8071-468e-a723-66dd43d02ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.699405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46cd4f-8071-468e-a723-66dd43d02ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.719973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6d46cd4f-8071-468e-a723-66dd43d02ab9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:30.721165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6d46cd4f-8071-468e-a723-66dd43d02ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:30.730439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 746/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:30.887043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b5ece-5ceb-42d1-a4ce-e3dc08bc5144 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:30.902772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b5ece-5ceb-42d1-a4ce-e3dc08bc5144 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:30.908441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:30.912299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:30.912299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:30.912299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:30.918594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:30.920255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996ee87-471f-4bcf-9153-5ca6214a22b3 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:30.931111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 778/1524] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:31.766643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49f4b-94ec-4336-a9ba-c28e31abaa9a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:31.767209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49f4b-94ec-4336-a9ba-c28e31abaa9a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:31.767209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d49f4b-94ec-4336-a9ba-c28e31abaa9a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:31.768576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 747/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 21:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:31.773491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639767b1-43dc-45aa-93c4-ad2b777384e8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:31.773935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639767b1-43dc-45aa-93c4-ad2b777384e8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:31.774077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639767b1-43dc-45aa-93c4-ad2b777384e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:31.794285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-639767b1-43dc-45aa-93c4-ad2b777384e8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:31.795488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-639767b1-43dc-45aa-93c4-ad2b777384e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:31.803968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 779/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 21:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:31.960503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d49f4b-94ec-4336-a9ba-c28e31abaa9a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:31.976872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d49f4b-94ec-4336-a9ba-c28e31abaa9a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:31.982995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:31.983594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:31.983867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:31.984132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:31.992747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:31.994338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc57d2-9663-433f-ab30-24c174b93671 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:32.003562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 748/1527] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:36.481755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:36.482354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:36.482354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:36.483636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 780/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:36.489194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517c43-0c43-4c0b-9226-bb1bd7cb0fea None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:36.489664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517c43-0c43-4c0b-9226-bb1bd7cb0fea None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:36.489801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517c43-0c43-4c0b-9226-bb1bd7cb0fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:36.512075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-76517c43-0c43-4c0b-9226-bb1bd7cb0fea None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:36.513364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76517c43-0c43-4c0b-9226-bb1bd7cb0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:36.521511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 749/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:36.838835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:36.840591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:36.916783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96348697-b645-47ce-bc17-2fcc5bb1f567 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:36.920424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:36.921035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:36.921312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:36.921580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:36.931409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:36.932871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc8fb2-2bc8-4c73-8250-5851a0a91127 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:36.942282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 781/1530] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:38.957887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cab4-f5f9-4ae0-8e53-4f3ba13cdf2e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:38.958401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cab4-f5f9-4ae0-8e53-4f3ba13cdf2e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:38.958401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cab4-f5f9-4ae0-8e53-4f3ba13cdf2e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:38.959808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 750/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:38.965135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f3ab-919a-4495-b2bb-728e9dee6dfe None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:38.965626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f3ab-919a-4495-b2bb-728e9dee6dfe None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:38.965762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f3ab-919a-4495-b2bb-728e9dee6dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:38.986670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-22d8f3ab-919a-4495-b2bb-728e9dee6dfe None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:38.987965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-22d8f3ab-919a-4495-b2bb-728e9dee6dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:38.996563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 782/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:39.288870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513cab4-f5f9-4ae0-8e53-4f3ba13cdf2e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:39.307368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513cab4-f5f9-4ae0-8e53-4f3ba13cdf2e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:39.311002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:39.311608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:39.311879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:39.312145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:39.322040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:39.323484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032b458-8947-4f7b-9368-c9bd572bc619 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:39.333370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 751/1533] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:43.132264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904de00-0408-4013-9751-bcb407898af6 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:43.132885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904de00-0408-4013-9751-bcb407898af6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:43.132885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904de00-0408-4013-9751-bcb407898af6 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:43.134335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 783/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:22:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:43.140097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c59-54d1-4f51-a279-9b8a546713b1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:43.140604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c59-54d1-4f51-a279-9b8a546713b1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:43.140714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b51c59-54d1-4f51-a279-9b8a546713b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:43.161354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-46b51c59-54d1-4f51-a279-9b8a546713b1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:43.163207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46b51c59-54d1-4f51-a279-9b8a546713b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:43.171298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 752/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:22:43.747233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904de00-0408-4013-9751-bcb407898af6 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:43.749002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4904de00-0408-4013-9751-bcb407898af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:43.824399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904de00-0408-4013-9751-bcb407898af6 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:43.828204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:43.828855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:43.829134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:43.829399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:43.839641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:43.841139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dc88a-fe1b-44d1-98e4-b88666753cba service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:43.852401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 784/1536] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:22:45.950646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:45.951254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:45.951254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:45.952435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 753/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 21:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:22:45.958227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714134f8-9ada-43a4-83e3-9b71a67380d4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:45.958880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714134f8-9ada-43a4-83e3-9b71a67380d4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:45.958880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714134f8-9ada-43a4-83e3-9b71a67380d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:45.979964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-714134f8-9ada-43a4-83e3-9b71a67380d4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:22:45.981187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-714134f8-9ada-43a4-83e3-9b71a67380d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:45.989303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 785/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:22:46.569474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:22:46.571233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:22:46.584426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f81625-41f4-4912-a0c6-65a12703cbbd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:22:46.587975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:22:46.588568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:22:46.588841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:22:46.589106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:22:46.598058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:22:46.600184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddb91e-8bda-4c1e-963a-e296ccc22e45 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:22:46.612845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 754/1539] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:22:55.974404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=70 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:22:56.606247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=33 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:14.819891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:14.821113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:14.821627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:14.847321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:14.848916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:14.866098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 786/1540] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:14.875754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:14.876778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:14.877288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:14.904826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:14.906446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:14.918429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 755/1541] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:14.924105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:14.939756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5c26c-0686-41ae-a855-59ae45178425 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:14.945304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:14.946231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:14.946746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:14.947246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:14.948132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:14.948676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:14.950597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:14.970613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 787/1542] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:14.979308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:14.990183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028cc0b7-ce11-4b62-8de6-128ad760d0fa None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:14.998056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:14.999080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:14.999589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.000235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.001131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.003413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359dff7d-1b0f-48db-ae6f-ebd0907fdd64 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.111234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 756/1543] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:14 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:15.116175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.132514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ea11-038c-4d81-81e2-1a7b11974b5a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.221874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.223697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.224205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.230388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.231427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.233761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.361295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 788/1544] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:15.365489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55fc1-c585-43ee-bc58-8871e47eec3b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.366286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55fc1-c585-43ee-bc58-8871e47eec3b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.367100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f55fc1-c585-43ee-bc58-8871e47eec3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.374942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.392720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b45d6-651d-4180-a882-895bfe052a32 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.400339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d6f55fc1-c585-43ee-bc58-8871e47eec3b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.402333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d6f55fc1-c585-43ee-bc58-8871e47eec3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.402742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.405745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.405745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.405745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.405745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.406632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.422478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 757/1545] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.430376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 789/1546] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:15.432421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.432944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.433430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.444139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.466084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77862f-2eb0-40f7-8d73-9bb886d5adee admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.489105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.489635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.489951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.490219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.490879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.491201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.492702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.492948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.513374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 758/1547] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.519679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.533974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 790/1548] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:24:15.534682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa5df-d0b2-4f08-8ab2-7f4cf388ab72 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.538837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.542563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.543196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.543475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.543746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.544548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:15.544852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.549954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:15.549954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:24:15.549954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:24:15.551561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.559978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aa0ef-10e4-45f5-a3f6-b3f29ac1aa52 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.568581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.569269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.569546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.569844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] PATH_INFO: `/v3/projects/4b972cee2d99491096a0aba46391cec6/users/0b8cfb5d2a34451bb21d47bd5aae44f4/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.576568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 759/1549] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:15.586004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.601232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831a9b-998f-4dcf-b1d5-2c865e892196 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.609518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.610355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.610640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.611020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.612216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.619152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa101d-406f-4ee4-a2f8-87dc739449d4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.626685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b972cee2d99491096a0aba46391cec6, user_id=0b8cfb5d2a34451bb21d47bd5aae44f4, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.627813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:15.627813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:24:15.627813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:24:15.629806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5083ac-dc42-4ee2-b434-6c3db0b92004 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.731277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 791/1550] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:15 2022] PUT /identity/v3/projects/4b972cee2d99491096a0aba46391cec6/users/0b8cfb5d2a34451bb21d47bd5aae44f4/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:15.732687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.732977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.733250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.771222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 760/1551] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/projects => generated 391 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:24:15.772824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548b0d2-57c2-49e0-85d0-73f48f7958d9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.773166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548b0d2-57c2-49e0-85d0-73f48f7958d9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.773925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548b0d2-57c2-49e0-85d0-73f48f7958d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.776664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.777633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.808756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d548b0d2-57c2-49e0-85d0-73f48f7958d9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.810172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d548b0d2-57c2-49e0-85d0-73f48f7958d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.884201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 792/1552] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.885237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:15.898026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e045a-d8a9-4ad9-82e4-572d329d0c86 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:15.899374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 761/1553] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.900878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01890-1bb3-4760-9121-6101886fb2e6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.901168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01890-1bb3-4760-9121-6101886fb2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.901536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b01890-1bb3-4760-9121-6101886fb2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.903844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:15.904678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.905183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.906909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.906909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:15.909189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:15.927856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f6b01890-1bb3-4760-9121-6101886fb2e6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.931790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f6b01890-1bb3-4760-9121-6101886fb2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.952116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 762/1554] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.952116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88573626-3000-4be7-baa8-86bc52242a39 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.952116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88573626-3000-4be7-baa8-86bc52242a39 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.952116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88573626-3000-4be7-baa8-86bc52242a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:15.974454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-88573626-3000-4be7-baa8-86bc52242a39 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:15.975876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-88573626-3000-4be7-baa8-86bc52242a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:15.984402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 763/1555] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:15.988062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c326-2b84-4b9e-938a-3aa98669a399 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:15.988062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c326-2b84-4b9e-938a-3aa98669a399 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:15.988062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563c326-2b84-4b9e-938a-3aa98669a399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.007445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8563c326-2b84-4b9e-938a-3aa98669a399 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.008661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8563c326-2b84-4b9e-938a-3aa98669a399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.016726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 764/1556] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.019170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0606d-e3f5-4dd1-b4da-21469db9e3f9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.019170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0606d-e3f5-4dd1-b4da-21469db9e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.019170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0606d-e3f5-4dd1-b4da-21469db9e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.022634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 793/1557] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:15 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:16.022634 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.023208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.037900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:16.045746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ffa0606d-e3f5-4dd1-b4da-21469db9e3f9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.045746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ffa0606d-e3f5-4dd1-b4da-21469db9e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.066154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 765/1558] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.066154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7245d9-00cf-4711-82dd-46074a8558c7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.066154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7245d9-00cf-4711-82dd-46074a8558c7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.066154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7245d9-00cf-4711-82dd-46074a8558c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.086435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-af7245d9-00cf-4711-82dd-46074a8558c7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.087654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af7245d9-00cf-4711-82dd-46074a8558c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.107117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 766/1559] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.107679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ce84c-2d9e-4989-a70a-1c86243a0797 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.109528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.109943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.110215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.113822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.114449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.114727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.114997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.125257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.126749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8e4e-df63-4c21-a78f-6d068b723fe4 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.134755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.135932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.136877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 794/1560] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:16.138153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23884-b276-4f84-b887-d6d4b9be0fe4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.138420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23884-b276-4f84-b887-d6d4b9be0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.138676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23884-b276-4f84-b887-d6d4b9be0fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.146289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 767/1561] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.147091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.166570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8970f-433a-41e1-bf57-50e1f9688802 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.166712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f0d23884-b276-4f84-b887-d6d4b9be0fe4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.166712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f0d23884-b276-4f84-b887-d6d4b9be0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.166906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.166906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.166906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.166906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.167299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.167299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.174137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 795/1562] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.175509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc55c7d-6411-447e-bf2d-83747e690ca7 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.175837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc55c7d-6411-447e-bf2d-83747e690ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.176109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc55c7d-6411-447e-bf2d-83747e690ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.181660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 768/1563] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:16.181660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.192309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673baa00-47c5-47c5-bad3-76758a731405 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.197639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.198306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.198586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.198858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.198998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6dc55c7d-6411-447e-bf2d-83747e690ca7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.199506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.199811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.200057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6dc55c7d-6411-447e-bf2d-83747e690ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.201580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.209057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 796/1564] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.217859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb23f8-586f-414e-839d-cce9143c4d6b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.217859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb23f8-586f-414e-839d-cce9143c4d6b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.217859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb23f8-586f-414e-839d-cce9143c4d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.218187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 769/1565] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.218187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.226234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9e8be-b87f-403d-8bc4-51488540d635 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.231756 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.232366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.232636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.232900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.233234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-27cb23f8-586f-414e-839d-cce9143c4d6b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.233538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.233869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.234531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-27cb23f8-586f-414e-839d-cce9143c4d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.235463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.243856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 797/1566] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.245098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.245373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.245639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.252987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 770/1567] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.254047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.264852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d4d7-eb99-464f-8f16-d12a7661cbce admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.270183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.270804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.271074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.271338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.271981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.272400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.274003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.275649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:16.276051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.277272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.285835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 798/1568] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.293183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.293362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 771/1569] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.293362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.300041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fa5a5-45e1-4b5e-a8c2-1703c3983144 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.302649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa81da-0e04-4c2d-82a6-9a595e7fd9cb None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.309691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.309691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.309691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.309691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.310241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.310241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.310432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.310432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.310432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.310432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.310778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.311068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.313512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa39163-5dbd-4eea-86ed-71681dbda1a1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.322480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 772/1570] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.323398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.327475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 799/1571] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:16.328927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.329270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.330408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.344813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e47e90-efa9-42a9-b350-c731f89f7b83 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.359163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.359163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.359163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.359163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.359624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.359723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:16.359723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:16.359920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.375989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 800/1572] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:16.375989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.399461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfc0bf-7adc-4d22-8881-dc8772363e34 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.399461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.399461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.399461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:16.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:24:16.399933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:24:16.400337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.536372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 773/1573] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:16.537034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.598633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470fdf-51ab-4dbb-b448-6ef9954a2bdb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.605522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.608343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.608343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.608343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.608343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.608343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.609540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.613867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 801/1574] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:16.613867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.629632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 774/1575] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.629632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.643486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420050-ad9e-4c31-8a9e-2d0b7d9fb783 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.707638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b62dd-b740-4470-922f-0f7d21e09244 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.713328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.713984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.714256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.714835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.715196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.717370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.721450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.732774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.732774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.732774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.732774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.732774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.834897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 775/1576] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:16.835140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 802/1577] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:16.835140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.835331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.871231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:16.897626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:16.909797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b511a02-1500-4ce5-aecb-a3c07dd74bd6 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.914964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.915633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.915877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.916181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.916837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:16.917139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.918800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.927458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766238f9-c6fd-4ccd-9356-0ad8e6a16de3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.931801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 803/1578] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:16.932619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:16.941952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.942658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a63c87-db82-48a7-bd01-41178e392c92 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:16.943039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.943487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.943985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] PATH_INFO: `/v3/projects/786534cce098405196143139d375b465/users/dc0bc75b6b3f4d1689bc1f7faaa2e5bc/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.948877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:16.949502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:16.949823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:16.950105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:16.950738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.952684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:16.960713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786534cce098405196143139d375b465, user_id=dc0bc75b6b3f4d1689bc1f7faaa2e5bc, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:16.965319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.049888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 804/1579] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:16 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.050702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.060464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504eb14a-c1da-4a66-b463-54445b1ac297 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.065135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.065731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.066021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.066326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.066939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.068836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.083965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 776/1580] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:16 2022] PUT /identity/v3/projects/786534cce098405196143139d375b465/users/dc0bc75b6b3f4d1689bc1f7faaa2e5bc/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:17.089648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.147296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd948d-b582-49c6-bd6a-0b4feddc3340 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.152051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.152624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.152918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.153184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.153838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.155768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.181465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 805/1581] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.181465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.191228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1f86-78e8-4193-bb3c-1218f4593404 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.196188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.196892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.197170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.197462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.198129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.200548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.309363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 806/1582] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.310607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.315129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 777/1583] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.315129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.323668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31c072-8a03-4c83-ad51-631bf6adfc90 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.382662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772e9e6-fee4-4d26-a826-01469887d3f1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.387715 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.387883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.388351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.388609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.388765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.388864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.389217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.389479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.389598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.390231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.391430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.392178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbcb37-f6dc-493d-ba59-244074114b46 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.532071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 807/1584] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/users => generated 294 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.532909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.543939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f31298-6b28-4934-b17a-355f0ea4fdb9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.548772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.550060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.550338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.550608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.554086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 778/1585] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.554260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.554358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.554358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.554538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.554634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.564886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 808/1586] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.565853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.573376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:17.579706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:17.580352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509afb5e-5a66-4660-a496-da81b9e1bd36 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.614175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:17.625453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:17.638542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.639099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.643784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.643784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.643784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.643784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.649473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 779/1587] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:17.650363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.653308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 809/1588] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.654179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.661903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a14a45-b9a7-4076-b708-a3b681297934 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.664801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83fe6-b36e-49e4-9fe3-b1dc2dcb7e2c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.666487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.667094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.667365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.667632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.668266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.669633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.669888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.670367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.670643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.670912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.671536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.673472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.680298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 780/1589] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.681194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.691668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c29b06-d48c-4e93-b65e-e0959dd3faaf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.696726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.697322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.697599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.697910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.698594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.700218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.711108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 781/1590] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.712095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.726075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2e860-0089-4299-8035-3ed3b83aef06 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.731247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.731891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.732171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.732443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.733076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.734129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:17.734129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:24:17.734129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:24:17.735350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.803160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 810/1591] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.803160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.811268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985afeb-3d1d-4a23-968b-e153ccae9c6f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.817289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.817937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.818233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.818505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.819159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.820842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.837898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 811/1592] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.838524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.845766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 782/1593] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:17 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:17.846305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.852349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d1740-7c43-4557-98e8-7a1c91547054 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.860094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.861758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.862307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.862659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.863887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.866859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.879447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 812/1594] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.880677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.893286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fde29-4fd4-45ff-845d-a8fdd921bc40 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.899675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.900286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.901140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.901140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.901549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.903180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.911869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd622000-b3f5-43e8-b995-72f0f431fb8c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.917965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 813/1595] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.918452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.918963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.919098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.919339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.919675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.929944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ff956-d88e-4b4c-9176-8c3fac6057d7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.930589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.932100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.936039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.941527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.941527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.941527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.941527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.941527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.942871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 783/1596] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:17.943778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.952531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 814/1597] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.953755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408852a-8d21-47b2-aeb6-2e6b7d807c90 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.958050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:17.965846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.966550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.966859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.967330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.968563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:17.973314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:17.976357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a7e66-fa8b-4c8f-89d5-5e6962c73dfb admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:17.982053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:17.983558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:17.984261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:17.984705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] PATH_INFO: `/v3/projects/d1e4957407e948fc909aadade7d19719/users/2ca47c37cd2d4999afa598cac6daa2e7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:17.989767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 784/1598] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:17.990331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.001608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d551b42-7a33-4fd3-b578-362d06a61792 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.007278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e4957407e948fc909aadade7d19719, user_id=2ca47c37cd2d4999afa598cac6daa2e7, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.008494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.008919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.009255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.009588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.010333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.010582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.012166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.028344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 785/1599] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:18.029234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.040137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b56582-0cfc-4f73-bbfb-9e8a4747dd49 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.050360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.050360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.050360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.050360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] PATH_INFO: `/v3/projects/00d076788b1b429b80479174bf980939/users/a5d9eabfac7e4ca18cb69585cdb110bc/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.064015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d076788b1b429b80479174bf980939, user_id=a5d9eabfac7e4ca18cb69585cdb110bc, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.066862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.164943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 815/1600] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:17 2022] PUT /identity/v3/projects/d1e4957407e948fc909aadade7d19719/users/2ca47c37cd2d4999afa598cac6daa2e7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:18.166412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.243153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 786/1601] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:18 2022] PUT /identity/v3/projects/00d076788b1b429b80479174bf980939/users/a5d9eabfac7e4ca18cb69585cdb110bc/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:18.243153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.266885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbe087-e695-401f-9962-c842b2f1a2fd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.317950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:18.357401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe49ca-b00d-4a95-b1c0-46b6db5f7fc4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.365811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.366538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.366822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.367099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] PATH_INFO: `/v3/projects/dcbd2f3d11264077b82e63f3296f6d1f/users/c5cab93814fd4d4b8c7ce6ca806eecce/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.370146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.370640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.370914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.371196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] PATH_INFO: `/v3/projects/94e62daccf4b442c8ff2971abbedd74f/users/e4f87633619b42d989942e1d63d30fb0/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.393184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:18.401330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbd2f3d11264077b82e63f3296f6d1f, user_id=c5cab93814fd4d4b8c7ce6ca806eecce, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.407578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.416341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e62daccf4b442c8ff2971abbedd74f, user_id=e4f87633619b42d989942e1d63d30fb0, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.419066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.526903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 816/1602] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:18 2022] PUT /identity/v3/projects/dcbd2f3d11264077b82e63f3296f6d1f/users/c5cab93814fd4d4b8c7ce6ca806eecce/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:18.528177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.594417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 787/1603] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:18 2022] PUT /identity/v3/projects/94e62daccf4b442c8ff2971abbedd74f/users/e4f87633619b42d989942e1d63d30fb0/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:18.594417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.595260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc70c3-8fff-46f2-86a6-74098468f80b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.683987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.684760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.684893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.685153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.685832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.687593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f76c75-e2fc-4ddc-a059-c21e676a9727 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.688905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6941e52-05b2-40cc-9a28-79e15cc48f92 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.696286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.697041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.697327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.697595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.698351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.699327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 817/1604] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:18.700139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efea59-dc94-4f28-b6cf-62aef6fe22e1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.700987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538e67-e09a-437e-8ead-c7ff04166370 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.701278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538e67-e09a-437e-8ead-c7ff04166370 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.701559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49538e67-e09a-437e-8ead-c7ff04166370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.716613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 788/1605] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:18.719076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a9bc-cd21-4fa3-8a9c-368bddb7e6bb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.722108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a9bc-cd21-4fa3-8a9c-368bddb7e6bb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.722398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a9bc-cd21-4fa3-8a9c-368bddb7e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.726090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-49538e67-e09a-437e-8ead-c7ff04166370 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:18.727504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-49538e67-e09a-437e-8ead-c7ff04166370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:18.753471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 818/1606] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:18.753471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.753471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.764086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.777324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c436a9bc-cd21-4fa3-8a9c-368bddb7e6bb None None] MFA Rules not processed for user `a5d9eabfac7e4ca18cb69585cdb110bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:18.780693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c436a9bc-cd21-4fa3-8a9c-368bddb7e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:18.805916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] MFA Rules not processed for user `c5cab93814fd4d4b8c7ce6ca806eecce`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:18.807809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:18.889207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 789/1607] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:18.890871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.891184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.891406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.909519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 819/1608] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:18.910349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.922419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c07c2-28cf-4667-b472-3479f3eaf71c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:18.931141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:18.931830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:18.932169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:18.932459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] PATH_INFO: `/v3/projects/16ff8893c50a4d3a94968a3a2d235ba5/users/6abb91759fe44c699dae2254901bc658/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:18.934546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:18.950612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:18.960487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ff8893c50a4d3a94968a3a2d235ba5, user_id=6abb91759fe44c699dae2254901bc658, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:18.963201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:18.972778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 790/1609] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:18.973904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:18.997618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43889b7b-d6c8-4740-bada-e6cf775d5e02 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:19.007741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:19.008535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.008886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.009230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] PATH_INFO: `/v3/projects/c62a4d1d2a8f41c8971f271ed5f38908/users/b04de2af30e54ef385c8853e8bd47baf/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.115705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62a4d1d2a8f41c8971f271ed5f38908, user_id=b04de2af30e54ef385c8853e8bd47baf, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:19.115705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:19.116146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 820/1610] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:18 2022] PUT /identity/v3/projects/16ff8893c50a4d3a94968a3a2d235ba5/users/6abb91759fe44c699dae2254901bc658/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:19.116146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:19.160118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 791/1611] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:18 2022] PUT /identity/v3/projects/c62a4d1d2a8f41c8971f271ed5f38908/users/b04de2af30e54ef385c8853e8bd47baf/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:19.161077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:19.196364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453dae9-aaca-4f49-af87-a7a083eb5f54 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:19.206731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:19.254852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:19.284355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:19.285427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.285681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.286182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.286834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75b73c-d11b-406d-a8de-58e9e300d0e7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:19.298477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:19.298731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:19.298858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.299148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.299351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.299921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b371c6b-3de6-4ea9-ad51-7dea1f918392 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:19.300029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:19.302007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237276c-b74b-4745-82cf-e82b53dafb83 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:19.311325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 821/1612] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:19.313136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d832-b3b2-4f43-b0c2-1b991d5f381a None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.313136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d832-b3b2-4f43-b0c2-1b991d5f381a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.313362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32d832-b3b2-4f43-b0c2-1b991d5f381a None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.315496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 822/1613] 10.222.0.99 () {54 vars in 962 bytes} [Fri Sep 2 21:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:19.316596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32d832-b3b2-4f43-b0c2-1b991d5f381a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:19.328186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32d832-b3b2-4f43-b0c2-1b991d5f381a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:19.333258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:19.333968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.334249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.335065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.346343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:19.348161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd12e75-60bd-463c-9b22-9c7ea20bb39e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:19.361667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 823/1614] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:19.362999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d67f-67d0-4a02-abe5-5c0ac7603408 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.363279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d67f-67d0-4a02-abe5-5c0ac7603408 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.363550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d67f-67d0-4a02-abe5-5c0ac7603408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.399464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a644d67f-67d0-4a02-abe5-5c0ac7603408 None None] MFA Rules not processed for user `6abb91759fe44c699dae2254901bc658`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:19.400452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a644d67f-67d0-4a02-abe5-5c0ac7603408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:19.448426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 792/1615] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/projects => generated 381 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:19.450025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.450415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.450794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.498790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] MFA Rules not processed for user `b04de2af30e54ef385c8853e8bd47baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:19.501892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:19.507887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 824/1616] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:19.509464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:19.509766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:19.510045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:19.573396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:19.574847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:19.625401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 793/1617] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:19.625401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.195315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae4a0-2737-4906-9e5e-e794e0263445 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.204980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 825/1618] 10.222.0.99 () {58 vars in 1059 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:20.204980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.212055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.212055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.212055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.212055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.215175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.215175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8bbdc-97c2-4c82-9e64-c2d9425f6c95 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.220400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef9d15-56a8-428f-817a-10aedbf7e483 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.220400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.220400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.220400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.221587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.221587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.221587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 794/1619] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/users => generated 297 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd054bd-08ab-4d8e-a7bd-4f1769fd4120 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd054bd-08ab-4d8e-a7bd-4f1769fd4120 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd054bd-08ab-4d8e-a7bd-4f1769fd4120 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 795/1620] 10.222.0.99 () {54 vars in 962 bytes} [Fri Sep 2 21:24:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:20.222318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd054bd-08ab-4d8e-a7bd-4f1769fd4120 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.225304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd054bd-08ab-4d8e-a7bd-4f1769fd4120 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.225304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.225304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.225304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.226245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.226245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.227979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 826/1621] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:19 2022] POST /identity/v3/projects => generated 387 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:20.228484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b760-984d-465b-8686-1ce1734457c9 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.233838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.234417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 796/1622] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:20.234417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.234417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.234417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.266572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:20.268311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:20.329746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f8882-a48b-49f3-adae-78429529616a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.350847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.350847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.350847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.350847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.351184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.351184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.369003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 827/1623] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:20.370647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.387704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11477e14-ded1-4a69-9ba4-46a7c9857064 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.399905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.400931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.401562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.402076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.402976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.406016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.412468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 797/1624] 10.222.0.99 () {58 vars in 1059 bytes} [Fri Sep 2 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:20.416185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.447890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369f11c-4093-4d4f-b738-0327b2457439 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.459313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.460434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.460951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.462766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] PATH_INFO: `/v3/projects/96578deda46a4a988373913f9712b918/users/c83c7f1c514c4d8e9f6504c1762cf34e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.484660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96578deda46a4a988373913f9712b918, user_id=c83c7f1c514c4d8e9f6504c1762cf34e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.490021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.541089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 828/1625] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:20 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:20.541089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.556478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fe3b1-34bd-4cae-b2c3-e798bbb95e53 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.575794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.576997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.577285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.580170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.631019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 798/1626] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:20 2022] PUT /identity/v3/projects/96578deda46a4a988373913f9712b918/users/c83c7f1c514c4d8e9f6504c1762cf34e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:20.632140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.719872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.723753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.734003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570988e-9bb2-4a27-80b1-5acf0da1b24f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.739991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.740867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.741311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.741738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.742583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 829/1627] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:20.743892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.754681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.756501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2896a3-21a4-4b89-a43d-e8708e261ac3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.762436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa199bf-a37c-475e-bb00-019481feeba1 service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.768854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.770369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.770660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.771068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.771734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.773760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d3c84-fc75-431c-9f31-29ba26623467 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.776876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 799/1628] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:20.780989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a76d-fd3a-412d-8626-f3917495a76b None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.780989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a76d-fd3a-412d-8626-f3917495a76b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.780989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467a76d-fd3a-412d-8626-f3917495a76b None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.780989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 800/1629] 10.222.0.99 () {54 vars in 912 bytes} [Fri Sep 2 21:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:20.782128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467a76d-fd3a-412d-8626-f3917495a76b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.795564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 830/1630] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:20.796978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bfe6-6d0b-49c3-82e0-d2525b4b251f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.797298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bfe6-6d0b-49c3-82e0-d2525b4b251f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.797593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007bfe6-6d0b-49c3-82e0-d2525b4b251f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.801472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467a76d-fd3a-412d-8626-f3917495a76b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.807646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.808458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.808806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.809152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.810353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.812398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.825982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0007bfe6-6d0b-49c3-82e0-d2525b4b251f None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:20.829138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 801/1631] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:20.830043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0007bfe6-6d0b-49c3-82e0-d2525b4b251f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:20.831231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:20.844307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087cb50-ed63-48ef-a7cc-8ba2713a505a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:20.851132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:20.852904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:20.853292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:20.853581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] PATH_INFO: `/v3/projects/96578deda46a4a988373913f9712b918/users/c83c7f1c514c4d8e9f6504c1762cf34e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:20.857863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96578deda46a4a988373913f9712b918, user_id=c83c7f1c514c4d8e9f6504c1762cf34e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:20.861106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:20.879116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 802/1632] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:20 2022] PUT /identity/v3/projects/96578deda46a4a988373913f9712b918/users/c83c7f1c514c4d8e9f6504c1762cf34e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:20.888363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:21.000799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 831/1633] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:21.004309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.004746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.005461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.011122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668001d0-1193-4581-982f-882f0c5f76e7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:21.022708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:21.026221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.026531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.026862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] PATH_INFO: `/v3/projects/a7a0d33dc35b47dead0e31071daf2b10/users/feb4224a27ce4d2bbfa2a72b9e54d943/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.068034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a0d33dc35b47dead0e31071daf2b10, user_id=feb4224a27ce4d2bbfa2a72b9e54d943, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:21.072072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011766b5-8108-4fe1-9c4f-f94111b3f98d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:21.082667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] MFA Rules not processed for user `c83c7f1c514c4d8e9f6504c1762cf34e`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:21.083914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:21.180502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 832/1634] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:21.181464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:21.181874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 803/1635] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:20 2022] PUT /identity/v3/projects/a7a0d33dc35b47dead0e31071daf2b10/users/feb4224a27ce4d2bbfa2a72b9e54d943/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:21.185664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579249-ab33-4ddd-a912-2db6a7560d50 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.186263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579249-ab33-4ddd-a912-2db6a7560d50 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.189190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579249-ab33-4ddd-a912-2db6a7560d50 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.207213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 804/1636] 10.222.0.99 () {54 vars in 912 bytes} [Fri Sep 2 21:24:21 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:21.217706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.218452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.219278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.275092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] MFA Rules not processed for user `feb4224a27ce4d2bbfa2a72b9e54d943`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:21.276789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:21.323226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3551774-ecb6-4623-b016-b4112adc26bd None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:21.364944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:21.396904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:21.397756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.398051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.398422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.489371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 805/1637] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:21.490412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:21.511293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b15ad-558d-46c7-b93d-315ea7f2464f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:21.520551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:21.520949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:21.521536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.521854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.522122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.615727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:21.618632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb8cdf-708d-4f27-8657-45e2fe0adfe7 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:21.638204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 833/1638] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:21.639930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.640457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.641201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.667034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:21.669130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:21.681982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 834/1639] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:21.683118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:21.750720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:21.752241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:21.775338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 806/1640] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:21.881426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57353c-e465-46e0-8943-caa952ecb365 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:21.890206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:21.900021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:21.900021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:21.900021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:21.900021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:21.900021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:22.110656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:22.123356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5119b-45cd-4a12-8100-ec5ba9ae6cff service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:22.138191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:22.138947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:22.139260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:22.139851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:22.142350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 835/1641] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:21 2022] POST /identity/v3/projects => generated 387 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:22.147761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:22.166565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:22.169745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:22.189354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 807/1642] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:22.310304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8e13b-d76a-4893-8ad4-06dafdbc7742 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:22.319788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:22.324969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:22.324969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:22.324969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:22.324969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:22.330456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:22.511535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 836/1643] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:22 2022] POST /identity/v3/users => generated 300 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:22.512685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:22.529242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b93ba-161a-40fa-bfba-b48bb43bb153 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:22.542118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:22.542911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:22.543193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:22.543463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:22.544113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:22.545788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:22.579488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:22.581504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 808/1644] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:22.592154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b287161-5ec2-4434-88ce-eb4d22bd1b2b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:22.619150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:22.620275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:22.626315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:22.626937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] PATH_INFO: `/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0/users/8e92a0efc2844f3bb2475943e2127e2c/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:22.676577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5c2210dda24b108246a2bdca7ddfa0, user_id=8e92a0efc2844f3bb2475943e2127e2c, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:22.682070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:22.887831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 837/1645] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:22 2022] PUT /identity/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0/users/8e92a0efc2844f3bb2475943e2127e2c/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:22.891818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:22.906950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7fb0-f539-4518-8f0c-7e9bf556df4b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:23.113689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:23.114942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:23.115322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:23.115812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:23.124265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:23.132158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d2ab4-9a18-4ba0-8918-5e62e3ccc8d4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:23.186497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:23.204732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 809/1646] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:23.243667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213fd2fd-77e9-4ada-9413-00ce0bd8da1f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:23.255034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:23.259437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:23.260978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:23.262189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] PATH_INFO: `/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0/users/8e92a0efc2844f3bb2475943e2127e2c/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:23.267656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5c2210dda24b108246a2bdca7ddfa0, user_id=8e92a0efc2844f3bb2475943e2127e2c, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:23.277431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:23.323107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 838/1647] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:23 2022] PUT /identity/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0/users/8e92a0efc2844f3bb2475943e2127e2c/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:23.346017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:23.347389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:23.355634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:23.397108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] MFA Rules not processed for user `8e92a0efc2844f3bb2475943e2127e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:23.398693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:23.589391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:23.608776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 810/1648] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:23.769123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:23.770626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:23.863317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd05711-cd66-4341-8487-bacc4cbd829d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:23.884786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:23.886294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:23.890578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:23.891174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:23.911112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:23.925846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:23.927708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 839/1649] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:24.000400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:24.026156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb49d3-907e-48c2-bc96-da8ce83cc251 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:24.042943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:24.045696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:24.045882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:24.046652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:24.244892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:24.247936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e0445-8a6e-4ca2-afd0-02a6f34574ac service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:24.273566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 811/1650] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:25.793384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:25.950487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:26.226703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:26.241021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd5e6e-ea70-47fd-9b77-9a341a35ad66 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:26.245889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:26.246790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:26.247270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:26.247756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:26.420910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:26.425548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b82ee-ebd9-4bcc-bfca-bd43d7ed87ce service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:26.445582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 840/1651] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:26.836209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:29.473173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:29.477517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:29.477517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:29.524108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:29.549376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] MFA Rules not processed for user `0b8cfb5d2a34451bb21d47bd5aae44f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:29.551033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:29.689301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 812/1652] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:29.701161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7630d-186b-4043-8850-bd837b61c46a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:29.703564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7630d-186b-4043-8850-bd837b61c46a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:29.704316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7630d-186b-4043-8850-bd837b61c46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:29.753302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a2a7630d-186b-4043-8850-bd837b61c46a None None] MFA Rules not processed for user `0b8cfb5d2a34451bb21d47bd5aae44f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:29.757616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a2a7630d-186b-4043-8850-bd837b61c46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:29.776404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 841/1653] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:29.783572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:29.920408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92a9b6-2980-4bd5-8f29-1280181f76a8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:29.931868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:29.932519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:29.932799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:29.935677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:29.947654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:29.950119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4f229-6b5b-4974-aa54-0e454b6afde6 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:29.967471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 813/1654] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:31.227396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081db2-d074-42b8-af7d-cc3ad1f7312d None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:31.228335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081db2-d074-42b8-af7d-cc3ad1f7312d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:31.228929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53081db2-d074-42b8-af7d-cc3ad1f7312d None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:31.231331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 842/1655] 10.222.0.99 () {54 vars in 912 bytes} [Fri Sep 2 21:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:31.251740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6f647-e7b6-4e9c-b890-9f4d588eb7c6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:31.252356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6f647-e7b6-4e9c-b890-9f4d588eb7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:31.252356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6f647-e7b6-4e9c-b890-9f4d588eb7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:31.335830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-eec6f647-e7b6-4e9c-b890-9f4d588eb7c6 None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:31.335830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eec6f647-e7b6-4e9c-b890-9f4d588eb7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:31.438715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 814/1656] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:24:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:31.457592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081db2-d074-42b8-af7d-cc3ad1f7312d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:31.471484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53081db2-d074-42b8-af7d-cc3ad1f7312d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:31.479975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:31.480660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:31.480938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:31.481208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:31.506226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:31.508008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:31.527522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 843/1657] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:33.433096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd0a6-0ee1-44c0-9eb6-aac9cd8f84f8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:33.433684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd0a6-0ee1-44c0-9eb6-aac9cd8f84f8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:33.435022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd0a6-0ee1-44c0-9eb6-aac9cd8f84f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:33.485753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5b3fd0a6-0ee1-44c0-9eb6-aac9cd8f84f8 None None] MFA Rules not processed for user `2ca47c37cd2d4999afa598cac6daa2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:33.487219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b3fd0a6-0ee1-44c0-9eb6-aac9cd8f84f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:33.500693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:33.510405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:33.552589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:33.754132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a400f-3b91-48b8-8a98-63baef5a726e service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:33.761174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:33.762153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:33.762785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:33.763244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:33.766624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:33.785019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:33.789119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:33.845969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 844/1658] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:33.847278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 815/1659] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:33.863153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:33.909607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f6cd-f273-452e-b2a3-6616911e7a1e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:33.909607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f6cd-f273-452e-b2a3-6616911e7a1e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:33.910406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f6cd-f273-452e-b2a3-6616911e7a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:33.923180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555c54c-4e87-4860-92cc-0d65a498e1cf service placement] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:33.942886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:33.944776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:33.945061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:33.945331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:33.978557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:33.979124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2ad7f6cd-f273-452e-b2a3-6616911e7a1e None None] MFA Rules not processed for user `dc0bc75b6b3f4d1689bc1f7faaa2e5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:33.980745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86fb8d-4d5d-4b4c-9f50-a6e7bcac3faa service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:33.980887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2ad7f6cd-f273-452e-b2a3-6616911e7a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:34.018148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 845/1660] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:34.284281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 816/1661] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:34.284726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:34.285012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:34.285286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:34.346200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:34.346590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:34.346899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:34.364851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] MFA Rules not processed for user `dc0bc75b6b3f4d1689bc1f7faaa2e5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:34.379260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:34.407125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 846/1662] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:34.423164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] MFA Rules not processed for user `e4f87633619b42d989942e1d63d30fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:34.424826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:34.450897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:34.510893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c692372-6bad-4970-9c4b-2689e8385d41 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:34.552131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:34.552131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:34.552131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:34.552131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:34.583127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:34.587434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:34.622368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 847/1663] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:34.709373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 817/1664] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:34.748658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:34.765764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f70fc-68cb-40fd-80e4-ee2308c760e2 service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:34.772736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:34.773802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:34.773802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:34.774001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:34.790957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:34.792685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e6010-0a5e-4ffb-b2c9-9d8611697fd7 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:34.807474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 848/1665] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:35.844925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:36.006403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:36.152175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f345ca-6f87-4a36-9673-4089973b8866 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:36.182574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:36.183245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:36.183824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:36.184290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] PATH_INFO: `/v3/users/b04de2af30e54ef385c8853e8bd47baf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:36.185850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04de2af30e54ef385c8853e8bd47baf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:36.193439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:36.516826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7b94-d789-4016-bc95-75f05dc0e08f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:36.517501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7b94-d789-4016-bc95-75f05dc0e08f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:36.517881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f7b94-d789-4016-bc95-75f05dc0e08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:36.523196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04de2af30e54ef385c8853e8bd47baf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:36.524768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04de2af30e54ef385c8853e8bd47baf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:36.574858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-734f7b94-d789-4016-bc95-75f05dc0e08f None None] MFA Rules not processed for user `dc0bc75b6b3f4d1689bc1f7faaa2e5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:36.576178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-734f7b94-d789-4016-bc95-75f05dc0e08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:36.596480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 849/1666] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:36.634578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04de2af30e54ef385c8853e8bd47baf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:36.682773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04de2af30e54ef385c8853e8bd47baf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:36.754295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04de2af30e54ef385c8853e8bd47baf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:36.767181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 818/1667] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:35 2022] DELETE /identity/v3/users/b04de2af30e54ef385c8853e8bd47baf => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:36.769778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee49cb-579a-4651-b944-7209472143d3 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:36.771390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee49cb-579a-4651-b944-7209472143d3 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:36.771899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee49cb-579a-4651-b944-7209472143d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:36.799870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-faee49cb-579a-4651-b944-7209472143d3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:36.801682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-faee49cb-579a-4651-b944-7209472143d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:36.979113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 850/1668] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:37.045202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:37.052781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:37.460504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14be63-1c31-4891-83e9-b7d0dbd42ca0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:37.481248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:37.488441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:37.488441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:37.488441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:37.503052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:37.508212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:37.556420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 819/1669] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:38.395918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee49cb-579a-4651-b944-7209472143d3 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:38.397242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-faee49cb-579a-4651-b944-7209472143d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:38.430623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee49cb-579a-4651-b944-7209472143d3 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:38.452303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:38.453271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:38.458780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:38.458780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] PATH_INFO: `/v3/users/c5cab93814fd4d4b8c7ce6ca806eecce` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:38.460038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cab93814fd4d4b8c7ce6ca806eecce)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:38.461288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: WARNING py.warnings [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:38.461288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: warnings.warn(msg) Sep 02 21:24:38.461288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: Sep 02 21:24:38.465298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:38.984962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cab93814fd4d4b8c7ce6ca806eecce'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:38.985797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cab93814fd4d4b8c7ce6ca806eecce'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:38.987766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:39.037253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cab93814fd4d4b8c7ce6ca806eecce'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.163751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cab93814fd4d4b8c7ce6ca806eecce'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.166169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5d181833-5aa7-48f9-97f7-8b761842f414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cab93814fd4d4b8c7ce6ca806eecce'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.272711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:39.274401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:39.277970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 851/1670] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:38 2022] DELETE /identity/v3/users/c5cab93814fd4d4b8c7ce6ca806eecce => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:39.282396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:39.283108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:39.283617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:39.312573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dce6bb-57c9-4b64-ac45-9626bd3cb4cd service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:39.395088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:39.397571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:39.469408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:39.491856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:39.494535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:39.498198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:39.498579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] PATH_INFO: `/v3/projects/c62a4d1d2a8f41c8971f271ed5f38908` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:39.500001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62a4d1d2a8f41c8971f271ed5f38908)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:39.504574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: WARNING py.warnings [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 21:24:39.504574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: warnings.warn(msg) Sep 02 21:24:39.504574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: Sep 02 21:24:39.508184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:39.555712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 852/1671] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:39.567966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:39.570091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:39.709297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6ce2a-260f-48e4-a8d9-2e4d1abfa8d9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:39.718728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62a4d1d2a8f41c8971f271ed5f38908', 'request_id': 'req-c5be28d9-7eaf-40a3-8459-fe65069e2bde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.752222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62a4d1d2a8f41c8971f271ed5f38908', 'request_id': 'req-c5be28d9-7eaf-40a3-8459-fe65069e2bde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.826806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:39.828022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:39.828456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:39.829904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:39.883169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62a4d1d2a8f41c8971f271ed5f38908', 'request_id': 'req-c5be28d9-7eaf-40a3-8459-fe65069e2bde'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.902491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] The token cache is being invalidate because project c62a4d1d2a8f41c8971f271ed5f38908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:39.902491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.904599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c5be28d9-7eaf-40a3-8459-fe65069e2bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:39.933913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 820/1672] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:39 2022] DELETE /identity/v3/projects/c62a4d1d2a8f41c8971f271ed5f38908 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:39.935898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:39.936406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:39.938601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:39.945551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:39.947050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596c22b-d07d-4db8-8537-17f97d2c9b07 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:39.959891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 853/1673] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:39.976927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:39.978670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:39.994672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 821/1674] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:39.997044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:39.997332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:39.997607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.026174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:40.029638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:40.055631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.056032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 854/1675] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:40.080128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db6a59-67a2-40cd-ac1c-faeaff4047e4 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.089259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.090233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.090681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.091094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.091904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:40.092556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.095201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:40.111606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 822/1676] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:40.117232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.129093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fd905-e1e9-4ca4-a855-4cf02d79ec17 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.135648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.136555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.137263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.137646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.138649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.141088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:40.299212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 855/1677] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:40 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:40.299705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.318925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38667565-2a1a-4b11-a947-edf47d60b4ab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.595542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.599586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.600087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.600510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.601449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.604435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:40.732239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 823/1678] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:40 2022] POST /identity/v3/users => generated 288 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:40.736702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.767095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017c87c-862b-4abe-90ad-6c6d3d8d7172 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.775981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:40.779134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.780030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.780777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.781258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.782168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.785025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:40.814552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 856/1679] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:40.819863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.836801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9295fc-91f7-40db-9c61-7355d31c8266 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.866056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.868352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.869628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.870116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.872034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.874549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:40.929911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 824/1680] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:40.933827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.944906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:40.947890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:40.953653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc81e9-fee5-45af-9963-e1f4210edd75 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.971043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e10dc-5865-4cfa-a276-ee3efc1891f8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:40.978543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.982879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.984977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.985463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] PATH_INFO: `/v3/projects/1c985b90cb984d80b80295234a9cd915/users/bb85ca23ca83431bb495ffc779968bb8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.986476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:40.987146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:40.987479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:40.987748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] PATH_INFO: `/v3/projects/dcbd2f3d11264077b82e63f3296f6d1f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:40.990568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbd2f3d11264077b82e63f3296f6d1f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:40.995850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:41.032507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c985b90cb984d80b80295234a9cd915, user_id=bb85ca23ca83431bb495ffc779968bb8, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:41.036633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8e1e-4f3a-4b1f-8719-b64156b330be admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:41.183117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 857/1681] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:40 2022] PUT /identity/v3/projects/1c985b90cb984d80b80295234a9cd915/users/bb85ca23ca83431bb495ffc779968bb8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:41.184783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d9c3-c408-4da6-88d9-c6407d8ecf74 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:41.185216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d9c3-c408-4da6-88d9-c6407d8ecf74 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:41.185660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72d9c3-c408-4da6-88d9-c6407d8ecf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:41.214603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbd2f3d11264077b82e63f3296f6d1f', 'request_id': 'req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:41.220954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dc72d9c3-c408-4da6-88d9-c6407d8ecf74 None None] MFA Rules not processed for user `e4f87633619b42d989942e1d63d30fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:41.222980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dc72d9c3-c408-4da6-88d9-c6407d8ecf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:41.233407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbd2f3d11264077b82e63f3296f6d1f', 'request_id': 'req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:41.418310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbd2f3d11264077b82e63f3296f6d1f', 'request_id': 'req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:41.424129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 858/1682] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:41.425152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44473-4fa4-4921-ac1a-76dbe00d8031 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:41.426198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44473-4fa4-4921-ac1a-76dbe00d8031 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:41.426464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a44473-4fa4-4921-ac1a-76dbe00d8031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:41.470681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] The token cache is being invalidate because project dcbd2f3d11264077b82e63f3296f6d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:41.470978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:41.475469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d0ab206c-d5c7-4ff0-87d2-a13c0903f015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:41.478606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 825/1683] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:40 2022] DELETE /identity/v3/projects/dcbd2f3d11264077b82e63f3296f6d1f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:24:41.484628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-34a44473-4fa4-4921-ac1a-76dbe00d8031 None None] MFA Rules not processed for user `bb85ca23ca83431bb495ffc779968bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:41.487872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-34a44473-4fa4-4921-ac1a-76dbe00d8031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:41.488028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294454-e0b2-4b9d-8f16-66edbd20531b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:41.488028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294454-e0b2-4b9d-8f16-66edbd20531b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:41.488028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294454-e0b2-4b9d-8f16-66edbd20531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:41.515902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-15294454-e0b2-4b9d-8f16-66edbd20531b None None] MFA Rules not processed for user `2ca47c37cd2d4999afa598cac6daa2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:41.518068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-15294454-e0b2-4b9d-8f16-66edbd20531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:41.739548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 826/1684] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:41.742619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729042-28de-487a-9c54-a53fcf075729 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:41.743397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729042-28de-487a-9c54-a53fcf075729 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:41.749280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729042-28de-487a-9c54-a53fcf075729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:41.764043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 859/1685] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:41.834163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-28729042-28de-487a-9c54-a53fcf075729 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:41.843907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28729042-28de-487a-9c54-a53fcf075729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:41.965314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:42.122388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 827/1686] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:42.130167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.130909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.131488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.188692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:42.190509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:42.202955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 860/1687] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:42.208152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729042-28de-487a-9c54-a53fcf075729 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:42.225151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729042-28de-487a-9c54-a53fcf075729 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:42.239032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:42.239770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.240062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.240331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.241309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:42.242028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:42.245110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:42.273022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 828/1688] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:42.279719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:42.312385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:42.313682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36d50e-2518-469e-9494-67d0216c059c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:42.316416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:42.330126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:42.333203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.334336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.340389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.340389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:42.347556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:42.560722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 861/1689] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:42 2022] POST /identity/v3/projects => generated 373 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:42.569599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:42.591235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0b78e-3813-4ea7-a158-4657581df207 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:42.676169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:42.676169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.676169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.676169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.676740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:42.776150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c873-275f-4763-8287-f870d78a7672 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:42.781222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:42.782184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.782484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.782760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.783417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:42.785388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:42.789361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:42.791559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc877a-77dc-4e03-ba0e-d90549b7d21c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:42.806066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 829/1690] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:42.840814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7268e-f978-4662-a5c5-a39b0c2824bf None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.841269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7268e-f978-4662-a5c5-a39b0c2824bf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.841648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7268e-f978-4662-a5c5-a39b0c2824bf None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.843752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 830/1691] 10.222.0.99 () {54 vars in 912 bytes} [Fri Sep 2 21:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:42.852597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.852597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.852597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.892462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:42.906313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 862/1692] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:42 2022] POST /identity/v3/users => generated 287 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:42.908152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:42.909483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:42.955625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f63b-e84b-4860-90e1-c5bd64996fc8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:42.983143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:42.983798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:42.984084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:42.984362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:42.989501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:42.991648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:43.033469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 863/1693] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:24:43.034977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:43.065446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070938e-c3c7-4a0d-9918-2f12d5500b0a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:43.086524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:43.087227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.088219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.088219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.088496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:43.088959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 831/1694] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:24:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:43.102492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:43.111624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:43.114087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:43.123071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 864/1695] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:24:43.132241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643baf9-6dd7-4b58-84a3-98d10d724029 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:43.142039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:43.142443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:43.142894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.143197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.143476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.156873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:43.158746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96981a5-503e-4bdd-bf3f-0043523e36fa service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:43.165939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9c84f-ccd3-49b2-be3c-b003cf6b2a99 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:43.177771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:43.191371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.193514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.194337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] PATH_INFO: `/v3/projects/1713adfd6c434b29b739d7d80f50f5ab/users/a927fa5259b64a288acabf43aedc58b5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.207393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 832/1696] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:43.245430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1713adfd6c434b29b739d7d80f50f5ab, user_id=a927fa5259b64a288acabf43aedc58b5, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:43.252421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40898cdd-c7ca-4a60-b7d7-37a73f45f11c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:43.374283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:43.392809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 865/1697] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:43 2022] PUT /identity/v3/projects/1713adfd6c434b29b739d7d80f50f5ab/users/a927fa5259b64a288acabf43aedc58b5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:43.423950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.423950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.423950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.514570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.521988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.521988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.539478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] MFA Rules not processed for user `a927fa5259b64a288acabf43aedc58b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:43.542306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:43.720473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 833/1698] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:43.721094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:43.725965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:43.727738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:43.856777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbf554-b939-4fc9-a25c-124c2154cef1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:43.863692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:43.879125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:43.881339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:43.883985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:43.885164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:43.892758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:43.907102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 866/1699] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:24:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:44.120614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 834/1700] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:43 2022] POST /identity/v3/projects => generated 373 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:44.125639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:44.164684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb2433-5271-4c71-be7a-601e71a13f61 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:44.244798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:44.246341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:44.362597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:44.362597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:44.362597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:44.362597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:44.363800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:44.369948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:44.516857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250d42d-0ee5-42ec-a7c2-a57bead6b056 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:44.531225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:44.533901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:44.534197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:44.534535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:44.551450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 867/1701] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:44 2022] POST /identity/v3/users => generated 293 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:44.556287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:44.594435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5314-5d80-40d4-9ee8-da5d718e09b4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:44.613294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:44.613294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:44.613849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:44.614131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:44.614945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:44.621370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:44.655451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 868/1702] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:44.656938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:44.676929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:44.811969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:44.823979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:44.872216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 835/1703] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:44.874551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:44.919229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:44.921533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a6172-e0af-4e6d-a959-9598aa9c1ca8 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:44.956664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:44.970615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:44.971045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:44.971244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] PATH_INFO: `/v3/projects/99bc6f550ce1474eac845789a4f35f3c/users/1a778fbb39104aa6bc560e11cf87186e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:45.043962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bc6f550ce1474eac845789a4f35f3c, user_id=1a778fbb39104aa6bc560e11cf87186e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:45.053837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:45.109132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d913bba-2f25-4710-a3ed-0fd77aa2ecdb admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:45.150548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:45.154951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:45.155650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:45.156257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:45.223186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 836/1704] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:44 2022] PUT /identity/v3/projects/99bc6f550ce1474eac845789a4f35f3c/users/1a778fbb39104aa6bc560e11cf87186e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:45.226611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:45.398820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:45.398820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:45.452888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 869/1705] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:24:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:24:45.507652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57557c06-f88c-47e2-ad78-ea373efd1658 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:45.523014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:45.525764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:45.526408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:45.527126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:45.529241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:45.533853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b8f8f-5d32-4aef-8764-8fd7b13f3346 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:45.571910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:45.600441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 837/1706] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:45.605778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:45.713335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cf71e-d550-419e-bfe1-7bb97c362141 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:45.749672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:45.753521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:45.754066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:45.755582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] PATH_INFO: `/v3/projects/99bc6f550ce1474eac845789a4f35f3c/users/1a778fbb39104aa6bc560e11cf87186e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:45.766458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bc6f550ce1474eac845789a4f35f3c, user_id=1a778fbb39104aa6bc560e11cf87186e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:45.769222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:45.834447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 870/1707] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:45 2022] PUT /identity/v3/projects/99bc6f550ce1474eac845789a4f35f3c/users/1a778fbb39104aa6bc560e11cf87186e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:24:45.840905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:45.841855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:45.842275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:45.971858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] MFA Rules not processed for user `1a778fbb39104aa6bc560e11cf87186e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:45.979156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:46.152191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:46.152191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:46.409857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 838/1708] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 570 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:46.464003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:46.667905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1acc6-a97d-4ace-a5dc-8a2e49b520ab admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:46.674731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:46.676179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:46.676491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:46.676783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] PATH_INFO: `/v3/users/bb85ca23ca83431bb495ffc779968bb8` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:46.678866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb85ca23ca83431bb495ffc779968bb8)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:46.681427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:46.687056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62058d2-4190-4fa9-97b3-f014b2719cfc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:46.691509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:46.692312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:46.692727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:46.693282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:46.710742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:46.712811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833eede-a63b-4f5c-9ab5-bc4977b1fecd service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:46.732826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 839/1709] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:47.009562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb85ca23ca83431bb495ffc779968bb8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:47.019479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb85ca23ca83431bb495ffc779968bb8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:47.048151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb85ca23ca83431bb495ffc779968bb8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:47.149299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb85ca23ca83431bb495ffc779968bb8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:47.153595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb85ca23ca83431bb495ffc779968bb8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:47.201530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 871/1710] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:46 2022] DELETE /identity/v3/users/bb85ca23ca83431bb495ffc779968bb8 => generated 0 bytes in 1055 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:47.210761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:47.211362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:47.212653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:47.218382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:47.218883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:47.309982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:47.319670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:47.538713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71ca4e-9123-483e-a751-5ebdeae93fcf admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:47.563874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:47.566280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:47.566795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:47.567063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:47.569998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:47.612642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 840/1711] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:47.685659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:47.687781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:47.705130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 872/1712] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:47.719894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:47.721479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:47.871384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:47.876570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:47.891651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3003eb5-db94-40f7-b921-b8438f5ba8c3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:47.898259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:47.919143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:47.919143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:47.919143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:47.919143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:47.919143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8239bc1-3368-42f3-b013-363830655a41 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:47.919595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba1efa-514e-4d24-918b-6ac0b98ad29b service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:47.924196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:47.925101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:47.925424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:47.925744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] PATH_INFO: `/v3/users/a5d9eabfac7e4ca18cb69585cdb110bc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:47.932379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d9eabfac7e4ca18cb69585cdb110bc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:47.934340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 841/1713] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:47.935191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:48.320575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154ccf-7890-463c-a9f5-522c3e2ab048 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:48.337867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154ccf-7890-463c-a9f5-522c3e2ab048 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:48.337867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154ccf-7890-463c-a9f5-522c3e2ab048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:48.401245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d9eabfac7e4ca18cb69585cdb110bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:48.433158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d9eabfac7e4ca18cb69585cdb110bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:48.437490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-6c154ccf-7890-463c-a9f5-522c3e2ab048 None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:48.442253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6c154ccf-7890-463c-a9f5-522c3e2ab048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:48.456033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d9eabfac7e4ca18cb69585cdb110bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:48.479261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 842/1714] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Sep 2 21:24:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:48.564395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d9eabfac7e4ca18cb69585cdb110bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:48.565042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d9eabfac7e4ca18cb69585cdb110bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:48.651501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:48.665772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 873/1715] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:47 2022] DELETE /identity/v3/users/a5d9eabfac7e4ca18cb69585cdb110bc => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:48.670222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d791-378d-475d-8180-290e7cc9328d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:48.670509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d791-378d-475d-8180-290e7cc9328d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:48.671081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d791-378d-475d-8180-290e7cc9328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:48.679904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:48.682223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:48.731848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1824d791-378d-475d-8180-290e7cc9328d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:48.750100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1824d791-378d-475d-8180-290e7cc9328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:48.930897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 843/1716] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:48.938146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9d76b-544e-4e36-8026-9c7e2a19a383 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:48.945600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:48.946300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:48.946582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:48.946854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:48.957410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:48.961223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:48.979946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 874/1717] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:49.011748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824d791-378d-475d-8180-290e7cc9328d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:49.028545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824d791-378d-475d-8180-290e7cc9328d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:49.038463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:49.042203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:49.043890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:49.044447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:49.089322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:49.091982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:49.120317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 844/1718] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:49.126874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:49.131776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:49.146241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84171c49-328f-4677-91d7-b7fae9b29ce2 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:49.161835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:49.166042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:49.167905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:49.168983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:49.198899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:49.202013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098720a-bfb9-4a1f-9fa7-c6088dd282ec service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:49.264692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 875/1719] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:50.033512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:50.043474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:50.081360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3c50e-76d8-4b84-a0e6-3f8ba3d66431 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:50.102816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:50.103753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:50.104206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:50.104623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] PATH_INFO: `/v3/projects/1c985b90cb984d80b80295234a9cd915` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:50.106443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c985b90cb984d80b80295234a9cd915)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:50.109077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:50.370597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c985b90cb984d80b80295234a9cd915', 'request_id': 'req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:50.396680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c985b90cb984d80b80295234a9cd915', 'request_id': 'req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:50.560779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c985b90cb984d80b80295234a9cd915', 'request_id': 'req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:50.564010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:50.585926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] The token cache is being invalidate because project 1c985b90cb984d80b80295234a9cd915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:50.591133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:50.591133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:50.596106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 845/1720] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:50 2022] DELETE /identity/v3/projects/1c985b90cb984d80b80295234a9cd915 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:50.602092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac70a-6349-4308-8751-78ee978da079 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:50.603123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac70a-6349-4308-8751-78ee978da079 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:50.603476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac70a-6349-4308-8751-78ee978da079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:50.644372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7bcac70a-6349-4308-8751-78ee978da079 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:50.645694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7bcac70a-6349-4308-8751-78ee978da079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:50.738285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:50.740057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:50.834959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 876/1721] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:50.845552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:50.846075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:50.846744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:50.944567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:50.951183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:50.999312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 877/1722] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:51.000800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:51.032340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca28a2-0d9e-4f2b-901a-a462d5b0fcca admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:51.033270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af7b0b-2735-47e5-b5a8-8255818aa945 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:51.040221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:51.040856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.041133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.041401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.042719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:51.043121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:51.045210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:51.063940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:51.067291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.068884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.069607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] PATH_INFO: `/v3/users/6abb91759fe44c699dae2254901bc658` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.070282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 878/1723] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:24:51.074480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:51.079199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abb91759fe44c699dae2254901bc658)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:51.084284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:51.093015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a992089-bf0a-4091-9d57-69447005ea3b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:51.102822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:51.103906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.104545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.105378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.107102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:51.111897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:51.306025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 879/1724] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:51 2022] POST /identity/v3/projects => generated 405 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:24:51.308943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:51.433299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abb91759fe44c699dae2254901bc658'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:51.434193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abb91759fe44c699dae2254901bc658'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:51.470591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:51.539144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07b50-dd30-4431-87d5-25a6537c4b41 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:51.554756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:51.555756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.556164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.556545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.557422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:51.568191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:51.577399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abb91759fe44c699dae2254901bc658'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:51.601964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abb91759fe44c699dae2254901bc658'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:51.654829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dc927f15-1a32-4424-953f-a2c069da83c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abb91759fe44c699dae2254901bc658'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:51.674464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 846/1725] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:50 2022] DELETE /identity/v3/users/6abb91759fe44c699dae2254901bc658 => generated 0 bytes in 937 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:51.688949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.688949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.688949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.719045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 880/1726] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:51 2022] POST /identity/v3/users => generated 303 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:51.723163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:51.728513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:51.759031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:51.762795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:51.768736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da25172-f98e-4c54-9685-c369202917ac admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:51.901171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 847/1727] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:51.914910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:51.915916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:51.917217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:51.917217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:51.917559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:51.919917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:51.936586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 881/1728] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:51.947403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:51.991581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ddbe-7042-486e-8526-ee71c00696e7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:52.004938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:52.006001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.007240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.011783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.012975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:52.015009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:52.025160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:52.027735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:52.046167 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 848/1729] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:24:52.055267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:52.084039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad7637-2f87-4ff3-ab42-6adbfd8f6748 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:52.099033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:52.103805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.106019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.106529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] PATH_INFO: `/v3/projects/2c6c3d7a9b20465fa63d1146d92c23f2/users/eb54f23871764132be1051d13a508eb2/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.156478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6c3d7a9b20465fa63d1146d92c23f2, user_id=eb54f23871764132be1051d13a508eb2, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:52.162456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:52.174993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fd69-7787-48c1-88fe-6919fc519483 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:52.180658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:52.181597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.182172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.182596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.202445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:52.202445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba1d73-8610-45e8-94f2-ed3fb0c774bc service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:52.213734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 882/1730] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:24:52.329919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 849/1731] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:52 2022] PUT /identity/v3/projects/2c6c3d7a9b20465fa63d1146d92c23f2/users/eb54f23871764132be1051d13a508eb2/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:52.331722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.332010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.332300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.447350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] MFA Rules not processed for user `eb54f23871764132be1051d13a508eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:52.451475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:52.719973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 883/1732] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:52.721829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:52.868706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:52.872216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:52.908369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e24512-30bc-43ea-b9b0-9085d4823bfc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:52.916908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4f7ea-1789-45dd-8024-1238d2f23321 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:52.922260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:52.922770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:52.923430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.923900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.924243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] PATH_INFO: `/v3/projects/00d076788b1b429b80479174bf980939` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.925036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:52.925814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d076788b1b429b80479174bf980939)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:52.927829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:52.928281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:52.929918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:52.931087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:52.946696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:52.955629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:53.136673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 850/1733] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:52 2022] POST /identity/v3/projects => generated 403 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:53.138868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:53.208246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d076788b1b429b80479174bf980939', 'request_id': 'req-1b77a005-464b-465a-be3c-dc59efaf20c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.234560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d076788b1b429b80479174bf980939', 'request_id': 'req-1b77a005-464b-465a-be3c-dc59efaf20c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.321639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cc31f-6efb-4837-a5dd-19b262f72bfa admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:53.327793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:53.328739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:53.329175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:53.331035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:53.332024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:53.335941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde57a22-5891-41e5-8adb-e5d1364d8c74 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:53.380578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d076788b1b429b80479174bf980939', 'request_id': 'req-1b77a005-464b-465a-be3c-dc59efaf20c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.405441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] The token cache is being invalidate because project 00d076788b1b429b80479174bf980939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:53.406049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.413900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1b77a005-464b-465a-be3c-dc59efaf20c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.413900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 884/1734] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:52 2022] DELETE /identity/v3/projects/00d076788b1b429b80479174bf980939 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:53.413900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:53.413900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:53.413900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:53.417000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 885/1735] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:24:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:24:53.419133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:53.421984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:53.454794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d1267-a40b-4505-a9b9-ee2d501ea3f5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:53.465522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:53.467156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:53.467818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:53.468483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] PATH_INFO: `/v3/users/c83c7f1c514c4d8e9f6504c1762cf34e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:53.474770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83c7f1c514c4d8e9f6504c1762cf34e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:53.528276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 851/1736] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:53 2022] POST /identity/v3/users => generated 308 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:53.532123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:53.532332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:53.532632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:53.536151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:53.598079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:53.632333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:53.899824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83c7f1c514c4d8e9f6504c1762cf34e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.902590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83c7f1c514c4d8e9f6504c1762cf34e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.911949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 852/1737] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:24:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:53.916326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:53.922952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:53.946175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83c7f1c514c4d8e9f6504c1762cf34e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:53.999577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3698032-d76f-4361-8aa2-6cb4fe044c60 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:54.018243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:54.021024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.022223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.023324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.024926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:54.028663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:54.068263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83c7f1c514c4d8e9f6504c1762cf34e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:54.103553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83c7f1c514c4d8e9f6504c1762cf34e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:54.176268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 853/1738] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:54.191051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:54.200526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:54.205976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 886/1739] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:53 2022] DELETE /identity/v3/users/c83c7f1c514c4d8e9f6504c1762cf34e => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:54.206944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:54.208867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:54.296309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8762f0d-d3bc-4b27-bb3b-efb11fcbc4f0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:54.333594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:54.371588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11e9ae-9f80-4dba-8468-f2623e1a5855 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:54.383541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:54.383754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.384018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.384440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] PATH_INFO: `/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7/users/c4c328a5bc3f40afad71962405b41c3f/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.421996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07bd3b912b84d9ebe11e3bdb038e4c7, user_id=c4c328a5bc3f40afad71962405b41c3f, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:54.428414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:54.443589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:54.446041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.446725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.447211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.450609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:54.493412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:54.496063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:54.518980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 854/1740] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:54.530850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 887/1741] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:54 2022] PUT /identity/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7/users/c4c328a5bc3f40afad71962405b41c3f/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:54.535593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:54.555365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b2ea6-13a3-4f81-9c82-2f57ed9c6672 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:54.642802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:54.754888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:54.755721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.756005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.756279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.757760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:54.759703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614b521-858a-48c3-b4bd-540ad2850b80 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:54.793796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 855/1742] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:54.795726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:54.812933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b795a54-6b3b-4698-9fce-d1ba3106c29a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:54.820295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:54.821852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.822309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.822790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] PATH_INFO: `/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7/users/c4c328a5bc3f40afad71962405b41c3f/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.825749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07bd3b912b84d9ebe11e3bdb038e4c7, user_id=c4c328a5bc3f40afad71962405b41c3f, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:54.828984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:54.841806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 888/1743] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:54 2022] PUT /identity/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7/users/c4c328a5bc3f40afad71962405b41c3f/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:54.856521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:54.856934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:54.857388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:54.907833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] MFA Rules not processed for user `c4c328a5bc3f40afad71962405b41c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:54.909548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:55.103893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 856/1744] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:55.114340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:55.116425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:55.251193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9bfac-97f1-4d06-b351-0668605d1865 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:55.264091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:55.264091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:55.264091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:55.269438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:55.324685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:55.326947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:55.342075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 889/1745] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:24:55 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:24:55.587459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:55.589613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:55.720056 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06d753-7bb9-42a2-ba33-49132c383b59 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:55.725309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:55.726153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:55.726557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:55.727010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] PATH_INFO: `/v3/projects/16ff8893c50a4d3a94968a3a2d235ba5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:55.728350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ff8893c50a4d3a94968a3a2d235ba5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:55.730452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:55.983347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ff8893c50a4d3a94968a3a2d235ba5', 'request_id': 'req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:55.991420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ff8893c50a4d3a94968a3a2d235ba5', 'request_id': 'req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.139480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ff8893c50a4d3a94968a3a2d235ba5', 'request_id': 'req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.153774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] The token cache is being invalidate because project 16ff8893c50a4d3a94968a3a2d235ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:57.154460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.155945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.158129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 857/1746] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:55 2022] DELETE /identity/v3/projects/16ff8893c50a4d3a94968a3a2d235ba5 => generated 0 bytes in 1572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:57.162624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:57.166093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:57.168155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:57.174890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:57.194279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24282069-8531-4031-a5cd-220eaa187c3e service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:57.229275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:57.349361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:24:57.396658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6ff3c-1955-4e3c-a8c0-d2fbc74fc3f4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:57.415647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:57.421374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:57.421374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:57.421592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:57.421692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] PATH_INFO: `/v3/projects/96578deda46a4a988373913f9712b918` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:57.421807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:57.421807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:57.421807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] PATH_INFO: `/v3/users/feb4224a27ce4d2bbfa2a72b9e54d943` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:57.424195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb4224a27ce4d2bbfa2a72b9e54d943)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:57.425389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96578deda46a4a988373913f9712b918)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:57.428581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:57.438499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:57.779688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96578deda46a4a988373913f9712b918', 'request_id': 'req-65a0ae44-6238-432c-9d09-7eb3a982da7a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.788395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb4224a27ce4d2bbfa2a72b9e54d943'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.788919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96578deda46a4a988373913f9712b918', 'request_id': 'req-65a0ae44-6238-432c-9d09-7eb3a982da7a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.790853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb4224a27ce4d2bbfa2a72b9e54d943'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.923167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96578deda46a4a988373913f9712b918', 'request_id': 'req-65a0ae44-6238-432c-9d09-7eb3a982da7a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.935625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] The token cache is being invalidate because project 96578deda46a4a988373913f9712b918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:57.937932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.937932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-65a0ae44-6238-432c-9d09-7eb3a982da7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:57.938518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 890/1747] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:57 2022] DELETE /identity/v3/projects/96578deda46a4a988373913f9712b918 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:57.943356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e106a-9cb0-4dd3-9e8a-cdfbb472413d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:57.943558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e106a-9cb0-4dd3-9e8a-cdfbb472413d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:57.943973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e106a-9cb0-4dd3-9e8a-cdfbb472413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:57.966581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c38e106a-9cb0-4dd3-9e8a-cdfbb472413d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:57.971042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c38e106a-9cb0-4dd3-9e8a-cdfbb472413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:58.032024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 891/1748] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:58.041471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.045665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb4224a27ce4d2bbfa2a72b9e54d943'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:58.045811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.045811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.050698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb4224a27ce4d2bbfa2a72b9e54d943'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:58.066235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb4224a27ce4d2bbfa2a72b9e54d943'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:58.068497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:58.070385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:58.073936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 858/1749] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:57 2022] DELETE /identity/v3/users/feb4224a27ce4d2bbfa2a72b9e54d943 => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:58.086406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 892/1750] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:24:58.095612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.097260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:58.212276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98d34-8adf-40cf-bf07-44df2dda1b66 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.220639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.222459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.222953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.223444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.224291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:24:58.224759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.227575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.249973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 859/1751] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:24:58.253035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.282417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0f60-a56b-4277-9bf4-7de37a53f158 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.289188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.289865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.290394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.290830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.291829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.294008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.415959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 893/1752] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:58 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:24:58.420571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.434004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e7c2c-b7cb-4834-809b-1a784ef8f5d7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.512248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.513325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.513981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.514455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.515643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.519396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.730486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 860/1753] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:58 2022] POST /identity/v3/users => generated 282 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:58.734761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.745670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b578e0-463f-447c-86b9-78abafedad35 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.750888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.751508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.751788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.752067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.752785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.754467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.766706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 894/1754] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:24:58.771168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.776244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.779179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:58.785881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317661-ba60-461e-85c4-fcb49fda521c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.802925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6035b22-34b3-440d-bebe-e061af821b03 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.808758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.809558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.809683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.809849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.810137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.810772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.810964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.811348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.811798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] PATH_INFO: `/v3/projects/a7a0d33dc35b47dead0e31071daf2b10` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.812596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.813644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a0d33dc35b47dead0e31071daf2b10)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.816969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:58.828933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 861/1755] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:24:58.835008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:58.852061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc5180-d19c-4e75-a693-3b66655c803e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:58.861575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:58.862675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:58.863099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:58.863551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] PATH_INFO: `/v3/projects/3f77f25b06bc449a874fc40d8d872a5d/users/0d16fc1fdcc94a8c977cc38b68f57ce5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:58.895002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f77f25b06bc449a874fc40d8d872a5d, user_id=0d16fc1fdcc94a8c977cc38b68f57ce5, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:58.900729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b563e-4be1-43d4-b5fd-ed260c4c9544 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:59.019954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 862/1756] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:24:58 2022] PUT /identity/v3/projects/3f77f25b06bc449a874fc40d8d872a5d/users/0d16fc1fdcc94a8c977cc38b68f57ce5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:24:59.030195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a0d33dc35b47dead0e31071daf2b10', 'request_id': 'req-534d37ad-c780-4702-9f12-24db52685412'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.038362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a0d33dc35b47dead0e31071daf2b10', 'request_id': 'req-534d37ad-c780-4702-9f12-24db52685412'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.042876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:59.042876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:59.043126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:59.076406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] MFA Rules not processed for user `0d16fc1fdcc94a8c977cc38b68f57ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:24:59.077789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:59.173747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 863/1757] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:24:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:24:59.181601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:59.217359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a0d33dc35b47dead0e31071daf2b10', 'request_id': 'req-534d37ad-c780-4702-9f12-24db52685412'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.233798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] The token cache is being invalidate because project a7a0d33dc35b47dead0e31071daf2b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:24:59.234702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.235670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.237535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 895/1758] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:24:58 2022] DELETE /identity/v3/projects/a7a0d33dc35b47dead0e31071daf2b10 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:24:59.244959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:59.247726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:59.289570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07190bbe-9526-48d9-aa2c-4232e4e76bcf None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:59.297128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:59.297952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:59.298376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:59.298766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:59.299751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:59.302417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:59.341145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d37ad-c780-4702-9f12-24db52685412 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:59.348024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:59.348822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:59.349189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:59.349595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] PATH_INFO: `/v3/users/8e92a0efc2844f3bb2475943e2127e2c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:59.359118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e92a0efc2844f3bb2475943e2127e2c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:59.362329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:59.483173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 864/1759] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:24:59 2022] POST /identity/v3/projects => generated 363 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:24:59.483173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:59.486126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:59.639397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2658a-2dda-40d5-9ace-7f6f99d3e448 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:24:59.658987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:24:59.659930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:24:59.660412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:24:59.660869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] PATH_INFO: `/v3/users/eb54f23871764132be1051d13a508eb2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:24:59.662985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb54f23871764132be1051d13a508eb2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:24:59.665934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:24:59.678807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e92a0efc2844f3bb2475943e2127e2c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.680169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e92a0efc2844f3bb2475943e2127e2c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.687956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e92a0efc2844f3bb2475943e2127e2c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.827540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e92a0efc2844f3bb2475943e2127e2c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.829218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e92a0efc2844f3bb2475943e2127e2c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.856053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 896/1760] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:59 2022] DELETE /identity/v3/users/8e92a0efc2844f3bb2475943e2127e2c => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:24:59.857517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:24:59.859716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:24:59.958555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb54f23871764132be1051d13a508eb2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:24:59.962834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb54f23871764132be1051d13a508eb2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:00.050794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb54f23871764132be1051d13a508eb2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:00.068245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28790b-defa-47a7-94ea-9898ff970237 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:00.069437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb54f23871764132be1051d13a508eb2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:00.080601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:00.080825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:00.080825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:00.081532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:00.082196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:00.086712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:00.111739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-484da703-2e41-4dd3-8657-2a44685c6aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb54f23871764132be1051d13a508eb2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:00.127632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 865/1761] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:24:59 2022] DELETE /identity/v3/users/eb54f23871764132be1051d13a508eb2 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:00.130213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:00.138133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:00.138667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:00.202338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:00.203880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:00.227628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 897/1762] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:24:59 2022] POST /identity/v3/users => generated 288 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:00.233891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:00.237490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:00.287378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c653-586b-4444-ac17-93180f8fc2ac admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:00.369221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 866/1763] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:00.378343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:00.380856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:00.432428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:00.433330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:00.433330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:00.433614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:00.434287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:00.438938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:00.440684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:00.454028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 898/1764] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:00.460205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:00.498237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3fa19-dc7c-4972-8484-9792246ef9a0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:00.514846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:00.516563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:00.517158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:00.517806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] PATH_INFO: `/v3/projects/1e09c5ed734140899a20f4506f0d4fd2/users/057e7f258605455483a0c281c84ac6c4/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:00.625067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70593612-77d6-4e61-9e28-2b1687654b96 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:00.635582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e09c5ed734140899a20f4506f0d4fd2, user_id=057e7f258605455483a0c281c84ac6c4, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:00.638080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:00.640852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:00.641419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:00.642308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:00.644868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:00.666297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:00.668870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:00.697486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 867/1765] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:01.147752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 899/1766] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:00 2022] PUT /identity/v3/projects/1e09c5ed734140899a20f4506f0d4fd2/users/057e7f258605455483a0c281c84ac6c4/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:01.156786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:01.168173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305f0f-e7c8-48f0-912a-816ce4598f52 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:01.440546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:01.452834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:01.459821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:01.460202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:01.461895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:01.463767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:01.499905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 868/1767] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:01.504276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:01.536909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071a74-0197-4f68-badc-afe4f8a8e4e1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:01.551375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:01.567928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:01.573996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:01.573996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] PATH_INFO: `/v3/projects/1e09c5ed734140899a20f4506f0d4fd2/users/057e7f258605455483a0c281c84ac6c4/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:01.575926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e09c5ed734140899a20f4506f0d4fd2, user_id=057e7f258605455483a0c281c84ac6c4, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:01.583996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d606d00-ef74-4688-ae37-4cf09178a94b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:01.594763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:01.596728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:01.632254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 900/1768] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:01 2022] PUT /identity/v3/projects/1e09c5ed734140899a20f4506f0d4fd2/users/057e7f258605455483a0c281c84ac6c4/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:01.647849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff945e-487a-4aa4-994e-3af9f04883df admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:01.659407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:01.659891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:01.660340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:01.762043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] MFA Rules not processed for user `057e7f258605455483a0c281c84ac6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:01.764178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:01.820423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:01.861297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:01.862523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:01.863085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:01.863508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] PATH_INFO: `/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:01.881777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5c2210dda24b108246a2bdca7ddfa0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:01.886048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:02.068442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 901/1769] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:02.080530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:02.086722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:02.172112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5c2210dda24b108246a2bdca7ddfa0', 'request_id': 'req-6f791fce-72a2-4288-8a4e-08b345f0db35'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:02.199121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5c2210dda24b108246a2bdca7ddfa0', 'request_id': 'req-6f791fce-72a2-4288-8a4e-08b345f0db35'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:02.255296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c56e90-4f8a-4f86-9faf-9268f85380c7 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:02.288011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5c2210dda24b108246a2bdca7ddfa0', 'request_id': 'req-6f791fce-72a2-4288-8a4e-08b345f0db35'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:02.357442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] The token cache is being invalidate because project 6d5c2210dda24b108246a2bdca7ddfa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:02.359731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:02.362550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6f791fce-72a2-4288-8a4e-08b345f0db35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:02.366791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 869/1770] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:01 2022] DELETE /identity/v3/projects/6d5c2210dda24b108246a2bdca7ddfa0 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:02.376647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce8ce-47bf-4d78-81bc-4bbfc30105f4 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:02.377528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce8ce-47bf-4d78-81bc-4bbfc30105f4 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:02.378308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce8ce-47bf-4d78-81bc-4bbfc30105f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:02.379679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:02.381111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:02.381705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:02.382075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:02.384370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:02.476225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e72ce8ce-47bf-4d78-81bc-4bbfc30105f4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:02.480330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e72ce8ce-47bf-4d78-81bc-4bbfc30105f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:02.621279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 870/1771] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:02.631722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:02.633407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:02.634145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:02.676763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:02.680792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:02.682665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:02.683935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:02.720070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 871/1772] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:02.728323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:02.739741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 902/1773] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:02.755222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9c2c6-2fb9-4c25-a04a-950a7476e827 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:02.793364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:02.794045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:02.796906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:02.797187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:02.798066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:02.801917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:02.806179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:02.869381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 872/1774] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:25:02.871291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:02.894939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0674-f7fb-46ae-a18a-8b0c817f9572 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:02.921242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:02.921926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:02.922216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:02.922494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:02.933204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:02.937396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:03.067119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 903/1775] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:02 2022] POST /identity/v3/projects => generated 399 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:03.072632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:03.084762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785a49-ee6c-4273-b743-5b463a6a9104 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:03.166570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:03.255639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:03.306321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:03.308394 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:03.309623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:03.310802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:03.312891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:03.323929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:03.508848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 873/1776] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:03 2022] POST /identity/v3/users => generated 300 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:03.520989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:03.539556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd828-51d9-4213-8d56-1aba85196ccd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:03.556435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:03.557270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:03.561422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:03.562140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:03.573997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:03.575971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:03.591600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 904/1777] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:03.595615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:03.622429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74058-9b35-4fe2-91a2-910390970f2f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:03.627991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:03.628859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:03.629310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:03.629867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:03.630706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:03.632493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:03.649867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 874/1778] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:03.654035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:03.676822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55944fe-6c36-4102-b3f9-146112258fd8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:03.690690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:03.690690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:03.690690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:03.690690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] PATH_INFO: `/v3/projects/72002f8a24684147a78336cb3b5f9755/users/422116c5976644fdb7f5e6e402a4fe0d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:03.744129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=72002f8a24684147a78336cb3b5f9755, user_id=422116c5976644fdb7f5e6e402a4fe0d, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:03.744129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a1ed9-0abd-4d60-9609-c0879fdafd3a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:03.917016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:03.917016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:03.949941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 905/1779] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:03 2022] PUT /identity/v3/projects/72002f8a24684147a78336cb3b5f9755/users/422116c5976644fdb7f5e6e402a4fe0d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:03.952865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e94db-6f25-4b3b-a616-82c8f22e9502 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:03.979224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:03.979516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:03.979796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.018369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] MFA Rules not processed for user `422116c5976644fdb7f5e6e402a4fe0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:04.020362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:04.093120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:04.175291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:04.175291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.175291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.175291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] PATH_INFO: `/v3/projects/2c6c3d7a9b20465fa63d1146d92c23f2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.175809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6c3d7a9b20465fa63d1146d92c23f2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:04.177188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:04.214994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 906/1780] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:04.228151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:04.267429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dfaf3-e1a9-4ef8-a673-ecf00b2d33c4 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:04.278571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:04.279532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.279967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.280396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.283661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:04.289652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:04.346757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6c3d7a9b20465fa63d1146d92c23f2', 'request_id': 'req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:04.355730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6c3d7a9b20465fa63d1146d92c23f2', 'request_id': 'req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:04.438124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 907/1781] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:04 2022] POST /identity/v3/projects => generated 397 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:04.443914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:04.496199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6c3d7a9b20465fa63d1146d92c23f2', 'request_id': 'req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:04.514501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] The token cache is being invalidate because project 2c6c3d7a9b20465fa63d1146d92c23f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:04.515429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:04.516005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e34a7f2a-8e26-416f-9b7b-da6876f2ab7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:04.519536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 875/1782] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:03 2022] DELETE /identity/v3/projects/2c6c3d7a9b20465fa63d1146d92c23f2 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:04.521077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.521366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.521634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.524083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 876/1783] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:25:04.524820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:04.532347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:04.548647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c2c23-1a6e-4246-889b-0b605dc9b93d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:04.555890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:04.556545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.558249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.560790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.563953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df36a1-b96a-49ce-9886-6ce636c32711 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:04.564708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:04.571188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70137b17-8039-48c7-b3ea-e1d72561af6c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:04.577690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:04.578921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.583530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.583530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] PATH_INFO: `/v3/users/c4c328a5bc3f40afad71962405b41c3f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.583530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c328a5bc3f40afad71962405b41c3f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:04.583530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:04.874393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 908/1784] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:04 2022] POST /identity/v3/users => generated 305 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:04.874393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:04.874393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:04.874393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:04.896176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:04.897425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:04.979669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 909/1785] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:25:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:04.980870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:04.982607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.000535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea0e6c-bd9e-4501-a11b-c198d3c6749a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:05.007016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:05.008215 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.008215 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.008597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] PATH_INFO: `/v3/users/0d16fc1fdcc94a8c977cc38b68f57ce5` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.010716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d16fc1fdcc94a8c977cc38b68f57ce5)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:05.012829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:05.016701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c328a5bc3f40afad71962405b41c3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.018277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c328a5bc3f40afad71962405b41c3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.100275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c328a5bc3f40afad71962405b41c3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.126891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c328a5bc3f40afad71962405b41c3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.166201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c328a5bc3f40afad71962405b41c3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.202928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 877/1786] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:04 2022] DELETE /identity/v3/users/c4c328a5bc3f40afad71962405b41c3f => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:05.207149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:05.209227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.296359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a7913-f7c8-472f-b372-4c493578bada admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:05.302932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:05.303561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.303842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.304113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.306458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:05.308570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:05.325296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 878/1787] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:05.326495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:05.328842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.332358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d16fc1fdcc94a8c977cc38b68f57ce5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.333429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d16fc1fdcc94a8c977cc38b68f57ce5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.350342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d16fc1fdcc94a8c977cc38b68f57ce5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.423518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d30d1a-a83e-4413-ae88-7e20fdd4d5f8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:05.431045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:05.431712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.431989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.432268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.434404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.456054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:05.459696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:05.468699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d16fc1fdcc94a8c977cc38b68f57ce5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.470493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-2e1dd1dc-c7a2-4a62-af5c-09bae3cbafa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d16fc1fdcc94a8c977cc38b68f57ce5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:05.493388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 879/1788] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:05.495917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:05.497910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.527299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a236b0-844b-41eb-8b5d-e6837e57fee4 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:05.529960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 910/1789] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:04 2022] DELETE /identity/v3/users/0d16fc1fdcc94a8c977cc38b68f57ce5 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:05.532376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.532376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.532910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.561256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:05.562604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.660530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 911/1790] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:05.673465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:05.675765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:05.689013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:05.690569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.691350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.691856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] PATH_INFO: `/v3/projects/892798b47bca4fabb35a0367ba9a82d1/users/fb36d7d56e36420e82d5f95daeb4062d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.721967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892798b47bca4fabb35a0367ba9a82d1, user_id=fb36d7d56e36420e82d5f95daeb4062d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:05.725238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:05.757149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a394ac-358d-4685-9e7d-e4a5047c0053 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:05.766300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:05.768094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:05.768377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:05.768652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:05.779687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:05.781116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:05.804608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 912/1791] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:05.890168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 880/1792] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:05 2022] PUT /identity/v3/projects/892798b47bca4fabb35a0367ba9a82d1/users/fb36d7d56e36420e82d5f95daeb4062d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:05.894668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:05.915859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eafb64-433b-420b-a8e5-28c6c544a86e service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:06.024998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:06.026540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.027460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.028269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.029267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:06.034036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d3733-cadd-43a8-8357-bcdb16a53bb8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:06.059171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 913/1793] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:06.063984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:06.081337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020d325-c26b-4bb9-b2ca-9c95250c41a8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:06.096870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:06.097584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.097919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.098192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] PATH_INFO: `/v3/projects/892798b47bca4fabb35a0367ba9a82d1/users/fb36d7d56e36420e82d5f95daeb4062d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.100918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892798b47bca4fabb35a0367ba9a82d1, user_id=fb36d7d56e36420e82d5f95daeb4062d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:06.103670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:06.127960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 881/1794] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:06 2022] PUT /identity/v3/projects/892798b47bca4fabb35a0367ba9a82d1/users/fb36d7d56e36420e82d5f95daeb4062d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:06.143954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.145110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.145858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.168591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:06.225168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] MFA Rules not processed for user `fb36d7d56e36420e82d5f95daeb4062d`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:06.226899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:06.340810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 914/1795] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:06.352651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:06.354804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:06.428848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f35d-c836-445f-970c-059063be5cc3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:06.433583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:06.434240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.434511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.434782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.444711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:06.447488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:06.458340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 882/1796] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:06.510658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:06.513247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:06.517390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:06.520348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:06.629779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6b121-8195-4800-8cdd-14e90f7d1d45 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:06.637420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:06.638119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338340-27e7-4060-87bf-2e6394825c64 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:06.638579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.639113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.639558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] PATH_INFO: `/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.641572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07bd3b912b84d9ebe11e3bdb038e4c7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:06.645232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:06.646450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:06.647538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:06.648186 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:06.648624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] PATH_INFO: `/v3/projects/3f77f25b06bc449a874fc40d8d872a5d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:06.650483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f77f25b06bc449a874fc40d8d872a5d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:06.653266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:06.798739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07bd3b912b84d9ebe11e3bdb038e4c7', 'request_id': 'req-d582613d-8a2d-4ef5-9a28-831abe4aadd2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.798972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f77f25b06bc449a874fc40d8d872a5d', 'request_id': 'req-24a7a8bf-3f6e-4545-8fd1-19671049cc02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.806501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f77f25b06bc449a874fc40d8d872a5d', 'request_id': 'req-24a7a8bf-3f6e-4545-8fd1-19671049cc02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.807018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07bd3b912b84d9ebe11e3bdb038e4c7', 'request_id': 'req-d582613d-8a2d-4ef5-9a28-831abe4aadd2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.887803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:06.935097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f77f25b06bc449a874fc40d8d872a5d', 'request_id': 'req-24a7a8bf-3f6e-4545-8fd1-19671049cc02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.956360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] The token cache is being invalidate because project 3f77f25b06bc449a874fc40d8d872a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:06.956893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.957660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:06.959629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 883/1797] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:06 2022] DELETE /identity/v3/projects/3f77f25b06bc449a874fc40d8d872a5d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:06.960655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:06.963524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.026369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07bd3b912b84d9ebe11e3bdb038e4c7', 'request_id': 'req-d582613d-8a2d-4ef5-9a28-831abe4aadd2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.038701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a8bf-3f6e-4545-8fd1-19671049cc02 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.040124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] The token cache is being invalidate because project a07bd3b912b84d9ebe11e3bdb038e4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:07.040752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.041420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.043286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 915/1798] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:06 2022] DELETE /identity/v3/projects/a07bd3b912b84d9ebe11e3bdb038e4c7 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:07.044099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.044292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.044981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.045268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.045825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.046845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.047516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.139730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582613d-8a2d-4ef5-9a28-831abe4aadd2 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.151158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.154204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7a658-0adf-43a3-8c1d-def577b07767 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.172892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 884/1799] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:07.175254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256dff-8ce1-4bcd-8d37-a3eccc1679b9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.175767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256dff-8ce1-4bcd-8d37-a3eccc1679b9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.176374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f256dff-8ce1-4bcd-8d37-a3eccc1679b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.207254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.207944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.208188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.208459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] PATH_INFO: `/v3/users/057e7f258605455483a0c281c84ac6c4` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.209665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057e7f258605455483a0c281c84ac6c4)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.212575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.213176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2f256dff-8ce1-4bcd-8d37-a3eccc1679b9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:07.215437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2f256dff-8ce1-4bcd-8d37-a3eccc1679b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.232012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 885/1800] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:07.240285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.240568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.240843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.263284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:07.265670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.287546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 886/1801] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:07.290290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.334908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfe914-191d-4c1b-b8eb-5c281b74c4ce None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.351296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.352639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.353063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.353347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.354190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:07.354738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.356910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.375825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 887/1802] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:25:07.381354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.400934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65058ce-649a-4d2a-9160-17623015731e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.407294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.408100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.408388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.408664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.409523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.412295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.569904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 888/1803] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:07 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:07.576299 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.590290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057e7f258605455483a0c281c84ac6c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.594835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057e7f258605455483a0c281c84ac6c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.610323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057e7f258605455483a0c281c84ac6c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.697465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a80c8-f83a-473f-9fe3-69b10e7b8091 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.702728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.703365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.703643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.703918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.704592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.706939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.708870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057e7f258605455483a0c281c84ac6c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.710401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057e7f258605455483a0c281c84ac6c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:07.745369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 916/1804] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:07 2022] DELETE /identity/v3/users/057e7f258605455483a0c281c84ac6c4 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:07.840449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 889/1805] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:07 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:07.842615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.844768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:07.935382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06313cb-9753-4157-a59a-4f01d0913883 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.940183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.940804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.941086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.941366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.942046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.943656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.954621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 917/1806] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:07.958910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:07.970151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaced73-cfba-4d58-82b3-04184521baf9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:07.975616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:07.979064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:07.979064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:07.979064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:07.979064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:07.980322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786ac9c-86c9-4920-b737-a0d6e6e65eaf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:07.995363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 890/1807] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:08.000912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.018792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62f71-f70a-4c48-822a-83e8b0afc970 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:08.026142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:08.026930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.027507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.027507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] PATH_INFO: `/v3/projects/377a2ea603f6470d85a36d9384d9e92e/users/d33b063bd44043a7a2f12be05e48b48b/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.058146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=377a2ea603f6470d85a36d9384d9e92e, user_id=d33b063bd44043a7a2f12be05e48b48b, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:08.061462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:08.210161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 918/1808] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:07 2022] PUT /identity/v3/projects/377a2ea603f6470d85a36d9384d9e92e/users/d33b063bd44043a7a2f12be05e48b48b/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:08.212098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.212098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.212535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.267947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] MFA Rules not processed for user `d33b063bd44043a7a2f12be05e48b48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:08.270051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:08.389552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 891/1809] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:08.394574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.416644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.418621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:08.474055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a885-dfc2-4be7-ac63-8be2a8b0eaf1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:08.480456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:08.480456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.481592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.481592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.481592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:08.483633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:08.503677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42806b66-5271-4f14-b0e8-b0ae74046a4e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:08.508047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:08.508912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.509255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.509529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] PATH_INFO: `/v3/projects/1e09c5ed734140899a20f4506f0d4fd2` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.510861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e09c5ed734140899a20f4506f0d4fd2)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:08.512821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:08.525405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:08.602099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 919/1810] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:08 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:08.623197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.691644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476406dc-9b50-4865-8d20-564ef4bf143a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:08.697331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:08.698378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.698653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.698924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.699580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:08.701527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:08.719370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e09c5ed734140899a20f4506f0d4fd2', 'request_id': 'req-367c8210-0801-488a-a18e-bdc5116664cd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:08.745631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e09c5ed734140899a20f4506f0d4fd2', 'request_id': 'req-367c8210-0801-488a-a18e-bdc5116664cd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:08.858617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e09c5ed734140899a20f4506f0d4fd2', 'request_id': 'req-367c8210-0801-488a-a18e-bdc5116664cd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:08.870080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 920/1811] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:08 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:08.876169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] The token cache is being invalidate because project 1e09c5ed734140899a20f4506f0d4fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:08.876535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:08.877295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-367c8210-0801-488a-a18e-bdc5116664cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:08.879012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 892/1812] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:08 2022] DELETE /identity/v3/projects/1e09c5ed734140899a20f4506f0d4fd2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:08.879542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.882639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:08.884447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ac6df-71e5-401c-b29d-04990b42b36d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.884733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ac6df-71e5-401c-b29d-04990b42b36d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.885027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ac6df-71e5-401c-b29d-04990b42b36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.897580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bafaf-5ced-4f8c-bbc2-669fd74a1750 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:08.906490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-be3ac6df-71e5-401c-b29d-04990b42b36d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:08.907740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-be3ac6df-71e5-401c-b29d-04990b42b36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:08.967819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:08.967819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:08.967819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:08.967819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:08.968269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:08.968269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:08.981681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 921/1813] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:08.985056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:08.988166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 893/1814] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:08.996034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a3c2b-af67-4188-bd76-c24765b53970 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.001143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.002066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.002198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.002495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.002586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.002902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.002989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] PATH_INFO: `/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa/users/fb3ac2f7137e46fa831023ac1883cf98/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.019502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a7e6d99c5d483aab8e56f9e7915aaa, user_id=fb3ac2f7137e46fa831023ac1883cf98, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.022513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.026940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:09.028173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:09.036457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 894/1815] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:09.041675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.050805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334b777-b2bb-4ea4-a7e5-ef6f2884412e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.056585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.057198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.057478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.059748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.059748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:09.059748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.060305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.075233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 895/1816] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:25:09.078230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.089448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4948e21-19de-4ee3-8c23-9ad620ea10fd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.096871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.097366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.097847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.098077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.105647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.105647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.159559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 922/1817] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:08 2022] PUT /identity/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa/users/fb3ac2f7137e46fa831023ac1883cf98/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:09.163722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.229582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a5093-2a4c-475b-8a99-3f4d1e5a4347 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.234506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.235124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.235404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.235707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.236323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.238050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.258023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 923/1818] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:09.266101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 896/1819] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:09 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:09.266321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.267221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.276258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c7668-e71e-46e9-93b4-a039873cea8d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.355690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d603a-5ae3-4b0f-a4ad-0d0ce944671e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.361096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.362157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.362633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.362978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.363712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.366098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.370666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.371663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.372017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.372458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] PATH_INFO: `/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa/users/fb3ac2f7137e46fa831023ac1883cf98/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.375172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a7e6d99c5d483aab8e56f9e7915aaa, user_id=fb3ac2f7137e46fa831023ac1883cf98, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.380619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078f06-7dcd-4738-883b-7d5673abaaf2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.396165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 924/1820] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:09 2022] PUT /identity/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa/users/fb3ac2f7137e46fa831023ac1883cf98/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:09.409412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.409802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.410193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.432144 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:09.470020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] MFA Rules not processed for user `fb3ac2f7137e46fa831023ac1883cf98`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:09.471519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:09.476188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 897/1821] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:09 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:09.485963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.502349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3bcc9-df3a-4f4a-b628-964c43ec9f69 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.610421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 925/1822] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:09.617468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.622032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:09.649678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.649678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.652782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.653130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.654526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.655896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.671414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 898/1823] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:09.675365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.697384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c6a4-14df-48d5-b6ae-956bd201f7aa admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.702910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.703778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.704132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.704692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.705358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.707025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.714896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638c4b7-b7cf-4cfc-9c1e-781216b4d2f2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.720076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 899/1824] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:09.723660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.724376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:09.724593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.725259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.725843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.729826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:09.750198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7130829-4143-4ab9-9610-6478a35b4180 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:09.756854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:09.758379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.759066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.759849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] PATH_INFO: `/v3/projects/c7c78429eb1144b191ae8599537c5321/users/ffe1992120bb42fcafc909f14b0f5fb3/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.789705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c78429eb1144b191ae8599537c5321, user_id=ffe1992120bb42fcafc909f14b0f5fb3, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.796326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.825052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:09.826581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc7b7b-00df-49d1-b3e9-066323183233 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:09.836054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 926/1825] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:09.934239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 900/1826] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:09 2022] PUT /identity/v3/projects/c7c78429eb1144b191ae8599537c5321/users/ffe1992120bb42fcafc909f14b0f5fb3/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:09.945378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:09.946186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:09.946482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:09.989794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] MFA Rules not processed for user `ffe1992120bb42fcafc909f14b0f5fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:09.991077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:10.084655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 927/1827] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:10.090522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:10.180121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e535a9-d61c-4139-b67f-cba90d55438a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:10.187403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:10.188142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:10.188422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:10.188693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:10.189342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:10.191464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:10.331871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 901/1828] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:10 2022] POST /identity/v3/projects => generated 367 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:10.336220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:10.348915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c4a1c-24c4-447a-8745-0fe817ea7e9d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:10.431248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:10.432319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:10.432810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:10.433301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:10.434513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:10.438612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:10.641517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 928/1829] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:10 2022] POST /identity/v3/users => generated 290 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:10.646054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:10.673782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb50df-accd-4ed2-8ed6-1a564c5671ce admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:10.683369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:10.685087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:10.685087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:10.685087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:10.686161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:10.689364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:10.708363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:10.716595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 902/1830] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:10.721321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eb5b5-7de7-4b68-87ca-f7acd7c6e743 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:10.732305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:10.733217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:10.733613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:10.734099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] PATH_INFO: `/v3/projects/11cc667c8b3849c9b6206331acfde612/users/49d766b66d8446f2bc9fdd50d340be3a/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:10.756404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cc667c8b3849c9b6206331acfde612, user_id=49d766b66d8446f2bc9fdd50d340be3a, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:10.760056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:10.939169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 929/1831] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:10 2022] PUT /identity/v3/projects/11cc667c8b3849c9b6206331acfde612/users/49d766b66d8446f2bc9fdd50d340be3a/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:10.953598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:10.965610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea74-7e4d-47cb-8eb7-d20dea186ea1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:11.063960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:11.064834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:11.065269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:11.065640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:11.066401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:11.068766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133945-447f-4fb1-8293-4e1fa538dc76 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:11.089238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 903/1832] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:11.089747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:11.103170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55920f71-e2a1-4354-bb48-4b9e5afbe543 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:11.112895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:11.114005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:11.114417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:11.116442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] PATH_INFO: `/v3/projects/11cc667c8b3849c9b6206331acfde612/users/49d766b66d8446f2bc9fdd50d340be3a/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:11.120882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cc667c8b3849c9b6206331acfde612, user_id=49d766b66d8446f2bc9fdd50d340be3a, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:11.127096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:11.137204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:11.152101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 930/1833] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:11 2022] PUT /identity/v3/projects/11cc667c8b3849c9b6206331acfde612/users/49d766b66d8446f2bc9fdd50d340be3a/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:11.173448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:11.174155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:11.174422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:11.189133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:11.231543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] MFA Rules not processed for user `49d766b66d8446f2bc9fdd50d340be3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:11.231770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:11.327577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 904/1834] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:11.345181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:11.465776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b2795-6418-4ce2-8cc2-fc0ad1563f2d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:11.465776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:11.465776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:11.465776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:11.466247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:11.477666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:11.479299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38828021-722e-45cc-9fd6-01c5fbd0a786 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:11.489324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 931/1835] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:11.770485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:11.782363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f27706-72e1-42ab-a38a-f92b8820a42f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:11.787735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:11.788673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:11.789235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:11.790038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:11.884943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:11.886801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076e9fd-cf3b-4287-b63e-e06d5254528d service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:11.901519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 905/1836] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:12.159279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:12.159959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:12.160450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:12.162445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 932/1837] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:25:12.185510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041821-481a-4256-9442-29cf530feaad None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:12.186116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041821-481a-4256-9442-29cf530feaad None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:12.186116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca041821-481a-4256-9442-29cf530feaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:12.219469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ca041821-481a-4256-9442-29cf530feaad None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:12.221502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ca041821-481a-4256-9442-29cf530feaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:12.346639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 906/1838] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:12.413626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:12.418592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:12.451878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca69efb-2665-4d47-9e60-c4379ec0a613 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:12.475400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:12.478979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:12.479423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:12.479833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:12.516404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:12.526477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439aea4-e40f-42fb-8978-81b627e97ed0 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:12.590185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 933/1839] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:14.286156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:14.391173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:15.081076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca041821-481a-4256-9442-29cf530feaad None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:15.085341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ca041821-481a-4256-9442-29cf530feaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:15.195817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca041821-481a-4256-9442-29cf530feaad None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:15.200768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:15.201412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:15.201683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:15.201974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] PATH_INFO: `/v3/users/422116c5976644fdb7f5e6e402a4fe0d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:15.203148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=422116c5976644fdb7f5e6e402a4fe0d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:15.205206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:15.481166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422116c5976644fdb7f5e6e402a4fe0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:15.482274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422116c5976644fdb7f5e6e402a4fe0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:15.666187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422116c5976644fdb7f5e6e402a4fe0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:15.676440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422116c5976644fdb7f5e6e402a4fe0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:15.705060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422116c5976644fdb7f5e6e402a4fe0d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:15.715261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 907/1840] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:15 2022] DELETE /identity/v3/users/422116c5976644fdb7f5e6e402a4fe0d => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:15.720033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:15.720599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:15.720599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:15.747251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:15.748531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:15.834820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 934/1841] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:15.846391 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:15.847858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:15.918395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a93928-5d5f-4cfb-a6eb-d1dde09bd021 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:15.922991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:15.923689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:15.924008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:15.924319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:15.935642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:15.937095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:15.947738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 908/1842] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:16.710863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:16.713342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:16.725441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85bc07-61c9-4980-ad22-a86b0887aa52 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:16.731074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:16.731721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:16.733147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:16.733147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] PATH_INFO: `/v3/projects/72002f8a24684147a78336cb3b5f9755` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:16.733942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72002f8a24684147a78336cb3b5f9755)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:16.735858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:16.920324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72002f8a24684147a78336cb3b5f9755', 'request_id': 'req-79c6a709-267a-483a-9974-c7201b903392'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:16.926831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72002f8a24684147a78336cb3b5f9755', 'request_id': 'req-79c6a709-267a-483a-9974-c7201b903392'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:16.981022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:17.007909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72002f8a24684147a78336cb3b5f9755', 'request_id': 'req-79c6a709-267a-483a-9974-c7201b903392'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.022807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] The token cache is being invalidate because project 72002f8a24684147a78336cb3b5f9755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:17.023168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.023913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.026384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 935/1843] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:16 2022] DELETE /identity/v3/projects/72002f8a24684147a78336cb3b5f9755 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:17.030840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:17.046883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee00f726-bec7-4dc3-bf76-63abca03285a service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:17.135011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:17.135399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:17.135678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:17.135946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] PATH_INFO: `/v3/users/fb36d7d56e36420e82d5f95daeb4062d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:17.137536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb36d7d56e36420e82d5f95daeb4062d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:17.139803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:17.347238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb36d7d56e36420e82d5f95daeb4062d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.347927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb36d7d56e36420e82d5f95daeb4062d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.454012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb36d7d56e36420e82d5f95daeb4062d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.462084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb36d7d56e36420e82d5f95daeb4062d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.476059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9a750af9-7bbd-4661-83f8-7314792a8155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb36d7d56e36420e82d5f95daeb4062d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:17.485824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 909/1844] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:17 2022] DELETE /identity/v3/users/fb36d7d56e36420e82d5f95daeb4062d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:17.614179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:18.160128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:18.161952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:18.177119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a709-267a-483a-9974-c7201b903392 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:18.289747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:18.291992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:18.292841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:18.295047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] PATH_INFO: `/v3/projects/892798b47bca4fabb35a0367ba9a82d1` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:18.297424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=892798b47bca4fabb35a0367ba9a82d1)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:18.303441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:18.467913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892798b47bca4fabb35a0367ba9a82d1', 'request_id': 'req-5bde0049-4228-4032-b54a-5bc6a3be7b4f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:18.474738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892798b47bca4fabb35a0367ba9a82d1', 'request_id': 'req-5bde0049-4228-4032-b54a-5bc6a3be7b4f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:18.608325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892798b47bca4fabb35a0367ba9a82d1', 'request_id': 'req-5bde0049-4228-4032-b54a-5bc6a3be7b4f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:18.621380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] The token cache is being invalidate because project 892798b47bca4fabb35a0367ba9a82d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:18.621680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:18.622567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5bde0049-4228-4032-b54a-5bc6a3be7b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:18.624309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 936/1845] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:18 2022] DELETE /identity/v3/projects/892798b47bca4fabb35a0367ba9a82d1 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:18.629615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:18.630281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:18.630281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:18.652055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:18.653695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:18.716148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 910/1846] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:18.724545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:18.724832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:18.725281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:18.746848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:18.748074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:18.756806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 937/1847] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:18.762427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:18.773353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd3b0d-58b6-40cc-8023-975cc351271e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:18.779105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:18.779738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:18.780016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:18.780280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:18.780941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:18.781242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:18.782915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:18.800268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 911/1848] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:25:18.800464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:18.808948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:18.818347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7dd36-c2d6-467f-b897-00d09c778c5f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:18.824542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:18.825404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:18.825685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:18.825997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:18.826639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:18.828754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:18.961459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 938/1849] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:18 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:18.964545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:18.975764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64fc8-ccb3-49d9-9468-4e25bc0d3568 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.057831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.058668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.058954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.059227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.059873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.061878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.158226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.160138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:19.211765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 912/1850] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:18 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:19.217107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.227639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6669-2e21-4d4a-922c-1cd972be31f5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.234644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.235506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.236004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.236457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.237293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.239111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.247552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e3629-5145-4bbf-9504-6cef0669afc5 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.251577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 913/1851] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:19.255898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.258071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.258713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.258999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.259271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.263303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:19.267270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b9ee8-6558-441c-837b-d1c11a1ea842 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.273013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.273796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.274062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.274425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.275242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.277399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.292492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 914/1852] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:19.297566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.317665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f055753-7f68-43ff-a79b-409ceaf3edd6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.324202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.325316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.325780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.326226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] PATH_INFO: `/v3/projects/0b6ba016de694fe783a9b6235eec944d/users/1e22b03199144741859f0149382d4e67/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.344981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6ba016de694fe783a9b6235eec944d, user_id=1e22b03199144741859f0149382d4e67, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.345509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.347345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbc6e5-817d-4d6c-93a5-4681f4a389f1 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.348141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.363400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 939/1853] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:19.458216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 915/1854] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:19 2022] PUT /identity/v3/projects/0b6ba016de694fe783a9b6235eec944d/users/1e22b03199144741859f0149382d4e67/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:19.471545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.471825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.472264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.523205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] MFA Rules not processed for user `1e22b03199144741859f0149382d4e67`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:19.524930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:19.578889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:19.644218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.644991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 940/1855] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:19.737482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd8be8-d383-4769-9880-a97a33026cbd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:19.764508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:19.765491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:19.766479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:19.766974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:19.767800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:19.771122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:19.958127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 916/1856] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:19 2022] POST /identity/v3/projects => generated 381 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:19.960618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:19.976975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ae65e-b0d2-4469-8661-20bf213b9443 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.039836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.040459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.040743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.041231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.041919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.044160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.184000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 941/1857] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:19 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:20.185219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:20.195583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312875ac-9ec4-495c-9f79-243a04a069bb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.200410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.202169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.202169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.202169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.202471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.203961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.215263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 917/1858] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:20.219735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:20.232362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607a48a-133c-4f3d-93c3-0c5695c2b486 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.237765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.238854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.238854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.239244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] PATH_INFO: `/v3/projects/76344879f078409ab575b106a6e8a084/users/0aa7564ee6db415a97f741a1b35ca936/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.257313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76344879f078409ab575b106a6e8a084, user_id=0aa7564ee6db415a97f741a1b35ca936, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.261535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.425740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 942/1859] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:20 2022] PUT /identity/v3/projects/76344879f078409ab575b106a6e8a084/users/0aa7564ee6db415a97f741a1b35ca936/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:20.425932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:20.437000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e192e-5f27-4f9a-8b11-ee2ec1faa30b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.509181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.511157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.511438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.511719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.512367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.514162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b5689-1992-4140-b16f-97c7851a9296 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.537069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 918/1860] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:20.550741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:20.578367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429765f-45c4-480e-aa57-ec00ddd16e15 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.584024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.584725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.585156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.594132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] PATH_INFO: `/v3/projects/76344879f078409ab575b106a6e8a084/users/0aa7564ee6db415a97f741a1b35ca936/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.594132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=76344879f078409ab575b106a6e8a084, user_id=0aa7564ee6db415a97f741a1b35ca936, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.594132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.607348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 943/1861] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:20 2022] PUT /identity/v3/projects/76344879f078409ab575b106a6e8a084/users/0aa7564ee6db415a97f741a1b35ca936/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:20.621882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.622146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.622433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.652634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] MFA Rules not processed for user `0aa7564ee6db415a97f741a1b35ca936`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:20.653890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:20.728317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 919/1862] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:20.739723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:20.799134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e3949-25f4-43b1-ad97-203d3afa05ab admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:20.802991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:20.803619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:20.803890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:20.804157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:20.814217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:20.815687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:20.826343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 944/1863] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:21.096833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:21.108305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ca69f-dbd5-46ee-a55a-fc816f24fde8 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:21.112523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:21.113147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:21.113430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:21.113701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:21.174363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:21.175844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee8ab6-63fb-4961-9682-c6c11acdeeb3 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:21.186728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 920/1864] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:21.500538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:21.502867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:21.574771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf29ef-068b-4e7e-8bc6-70f6b8435c4a service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:21.580730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:21.581377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:21.581657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:21.581954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] PATH_INFO: `/v3/users/ffe1992120bb42fcafc909f14b0f5fb3` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:21.583087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe1992120bb42fcafc909f14b0f5fb3)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:21.585024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:21.610541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:21.840507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe1992120bb42fcafc909f14b0f5fb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:21.840507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe1992120bb42fcafc909f14b0f5fb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:21.845168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe1992120bb42fcafc909f14b0f5fb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:21.928319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe1992120bb42fcafc909f14b0f5fb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:21.928943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe1992120bb42fcafc909f14b0f5fb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:21.949277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 945/1865] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:21 2022] DELETE /identity/v3/users/ffe1992120bb42fcafc909f14b0f5fb3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:21.954157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:21.954644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:21.954732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:21.998576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:21.999830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:22.070723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 921/1866] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:22.075057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:22.085539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:22.085539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:22.135611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:22.162236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce0d1f-9036-467a-9b1c-3b397adcc2ff admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:22.166047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:22.166665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:22.166939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:22.167209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:22.177044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:22.178578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:22.188852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 946/1867] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:22.673511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:22.673511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:22.690143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6a7e2-5611-4068-a0aa-b3e437e106c3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:22.713997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:22.713997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:22.713997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:22.713997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] PATH_INFO: `/v3/projects/c7c78429eb1144b191ae8599537c5321` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:22.718297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c78429eb1144b191ae8599537c5321)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:22.721447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:22.880523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c78429eb1144b191ae8599537c5321', 'request_id': 'req-118db13b-a080-416f-9161-2ca475e94541'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:22.887461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c78429eb1144b191ae8599537c5321', 'request_id': 'req-118db13b-a080-416f-9161-2ca475e94541'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.040309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c78429eb1144b191ae8599537c5321', 'request_id': 'req-118db13b-a080-416f-9161-2ca475e94541'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.057701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] The token cache is being invalidate because project c7c78429eb1144b191ae8599537c5321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:23.058625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.059898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.062535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 922/1868] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:22 2022] DELETE /identity/v3/projects/c7c78429eb1144b191ae8599537c5321 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:23.067083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:23.073606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:23.078897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5f469-5c00-4118-a1c8-ce9b236d69f2 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:23.147912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:23.148466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:23.148740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:23.149013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] PATH_INFO: `/v3/users/49d766b66d8446f2bc9fdd50d340be3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:23.150170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49d766b66d8446f2bc9fdd50d340be3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:23.152803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:23.438981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49d766b66d8446f2bc9fdd50d340be3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.439737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49d766b66d8446f2bc9fdd50d340be3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.448174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49d766b66d8446f2bc9fdd50d340be3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.529361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:23.574473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49d766b66d8446f2bc9fdd50d340be3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.575116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e43b3bf2-c3e8-490a-81f5-028068b3969d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49d766b66d8446f2bc9fdd50d340be3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:23.599327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 947/1869] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:23 2022] DELETE /identity/v3/users/49d766b66d8446f2bc9fdd50d340be3a => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:24.186717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:24.188455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:24.216902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db13b-a080-416f-9161-2ca475e94541 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:24.290224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:24.290883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:24.291166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:24.292093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] PATH_INFO: `/v3/projects/11cc667c8b3849c9b6206331acfde612` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:24.293317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11cc667c8b3849c9b6206331acfde612)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:24.295309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:24.402603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:24.414859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11cc667c8b3849c9b6206331acfde612', 'request_id': 'req-9f7cde41-ee92-45b0-9e05-725056b9cfa6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:24.422129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11cc667c8b3849c9b6206331acfde612', 'request_id': 'req-9f7cde41-ee92-45b0-9e05-725056b9cfa6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:24.531816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11cc667c8b3849c9b6206331acfde612', 'request_id': 'req-9f7cde41-ee92-45b0-9e05-725056b9cfa6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:24.546384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] The token cache is being invalidate because project 11cc667c8b3849c9b6206331acfde612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:24.546698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:24.547445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9f7cde41-ee92-45b0-9e05-725056b9cfa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:24.549018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 923/1870] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:24 2022] DELETE /identity/v3/projects/11cc667c8b3849c9b6206331acfde612 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:24.554848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:24.555325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:24.555767 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:24.586780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:24.588934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:24.670783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 948/1871] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:24.678688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:24.678972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:24.679296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:24.711121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:24.712368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:24.722405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 924/1872] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:24.726674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:24.745458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4177849-cb36-4cc7-b876-a9d9f8ea082a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:24.746896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:24.747856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:24.758175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:24.758175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:24.758175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:24.758175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:24.758175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:24.770153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 949/1873] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:25:24.774713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:24.785249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de699700-85a0-4fb7-9e1d-8eb6ea15f887 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:24.790213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:24.790852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:24.791135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:24.792025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:24.792728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:24.794696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:24.944937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 925/1874] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:24 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:24.950116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:24.960290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692940c2-57dd-4c0a-b601-9ec15e6fe3d0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.020137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.020660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.021458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.021458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.021871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.023784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.029570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:25.120744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 950/1875] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:24 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:25.125051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.136405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770d1e-b9ed-430b-b705-499246b6975a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.141129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.141773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.142055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.142326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.142977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.144616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.155850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 926/1876] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:25.162530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.173549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4c21e-bb0b-4edf-ad84-e138f840f9bf admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.178324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.178952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.179551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.179551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.180566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.182986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7c399-9289-45b6-9d17-12aa3829882f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.194882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 951/1877] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:25.200960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.212702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59961-172a-4d3f-aca1-adb5598cc3ef admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.219174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.219851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.220128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.220402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] PATH_INFO: `/v3/projects/d89296c0c8f74d37965e6deebb35f784/users/5861fd9b266641d896770ca13b2a1409/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.240211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89296c0c8f74d37965e6deebb35f784, user_id=5861fd9b266641d896770ca13b2a1409, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.245919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.321829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 927/1878] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:25 2022] PUT /identity/v3/projects/d89296c0c8f74d37965e6deebb35f784/users/5861fd9b266641d896770ca13b2a1409/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:25.335427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.335993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:25.336405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.336875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.367888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] MFA Rules not processed for user `5861fd9b266641d896770ca13b2a1409`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:25.369115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:25.454673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 952/1879] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:25.459300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.533883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791cd6e-fedd-40e3-834f-df08e8f702a1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.539924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.540535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.540817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.541087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.542481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.544301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.660449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 928/1880] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:25 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:25.665499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.676467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023112a6-5c17-4f6d-969a-9ba65c37af5e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.743779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.744563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.745283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.745597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.746282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.748267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.788736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:25.889911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 953/1881] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:25 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:25.898926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.909580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f43a4-dca4-410b-8dac-6d9e3d663c0f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.914295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.914907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.915189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.915459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.916107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.918404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:25.929564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 929/1882] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:25.934994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:25.944970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59865-6792-4a2f-82e4-d827c7d354c0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:25.951574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:25.952264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:25.952567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:25.952824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] PATH_INFO: `/v3/projects/91bc8faec2984d3398b98cebdd0b63a6/users/e47046f1bc8d43f3b15860144049b2a4/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:25.972016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=91bc8faec2984d3398b98cebdd0b63a6, user_id=e47046f1bc8d43f3b15860144049b2a4, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:25.975510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:26.102450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 954/1883] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:25 2022] PUT /identity/v3/projects/91bc8faec2984d3398b98cebdd0b63a6/users/e47046f1bc8d43f3b15860144049b2a4/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:26.108170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:26.125134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794e404-08cb-4422-a43c-8784b5621fef admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:26.189261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:26.189919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:26.190374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:26.190697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:26.192515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:26.193020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:26.194966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbc4f-3764-45f2-95c0-d3703e4f9a6d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:26.206108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 930/1884] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:26.210479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:26.222082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1c932-c51f-411b-afad-388aa7c457fb admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:26.227537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:26.228192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:26.228464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:26.228735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] PATH_INFO: `/v3/projects/91bc8faec2984d3398b98cebdd0b63a6/users/e47046f1bc8d43f3b15860144049b2a4/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:26.230795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91bc8faec2984d3398b98cebdd0b63a6, user_id=e47046f1bc8d43f3b15860144049b2a4, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:26.233483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:26.247091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 955/1885] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:26 2022] PUT /identity/v3/projects/91bc8faec2984d3398b98cebdd0b63a6/users/e47046f1bc8d43f3b15860144049b2a4/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:26.257942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:26.261588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:26.262028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:26.262369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:26.327805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] MFA Rules not processed for user `e47046f1bc8d43f3b15860144049b2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:26.330302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:26.410815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 931/1886] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:26.426112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:26.429488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:26.513010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae896f-fae7-435e-a589-d519b8336a3b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:26.517986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:26.519560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:26.520179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:26.520657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:26.534192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:26.536054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9410b49-5772-41d5-8d3a-7b9453ecee83 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:26.554195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 956/1887] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:26.612204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:26.678670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:27.104840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:27.249160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:27.315493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d43d3-2422-4613-ac0a-da629cc03fa1 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:27.326725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:27.327481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:27.327833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:27.328162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] PATH_INFO: `/v3/users/5861fd9b266641d896770ca13b2a1409` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:27.329399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5861fd9b266641d896770ca13b2a1409)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:27.333079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:27.647901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5861fd9b266641d896770ca13b2a1409'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:27.648600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5861fd9b266641d896770ca13b2a1409'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:27.663955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:27.723593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5861fd9b266641d896770ca13b2a1409'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:27.732425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5861fd9b266641d896770ca13b2a1409'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:27.744404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5861fd9b266641d896770ca13b2a1409'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:27.751295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 932/1888] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:27 2022] DELETE /identity/v3/users/5861fd9b266641d896770ca13b2a1409 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:27.756560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:27.757018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:27.757456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:27.786607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:27.788128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:27.884671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 957/1889] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:27.899428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:27.900528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:27.975458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050a43-226b-4cf6-88ac-ea6f7642c671 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:27.979216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:27.980345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:27.980345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:27.980536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:27.991020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:27.992476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:28.004125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 933/1890] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:28.703015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:28.704451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:28.706965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:28.717107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae7c7f-c4bb-4085-8761-df3b6c6d8a26 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:28.721927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:28.722552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:28.722824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:28.723091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] PATH_INFO: `/v3/projects/d89296c0c8f74d37965e6deebb35f784` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:28.724261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89296c0c8f74d37965e6deebb35f784)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:28.726224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:28.839552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89296c0c8f74d37965e6deebb35f784', 'request_id': 'req-e7bddb81-e36a-46ab-95bc-8010f195c27d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:28.845811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89296c0c8f74d37965e6deebb35f784', 'request_id': 'req-e7bddb81-e36a-46ab-95bc-8010f195c27d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:28.900758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:28.902315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:28.915284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa130219-92c8-4a48-9d87-14ec004957b4 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:28.931496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89296c0c8f74d37965e6deebb35f784', 'request_id': 'req-e7bddb81-e36a-46ab-95bc-8010f195c27d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:28.943255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] The token cache is being invalidate because project d89296c0c8f74d37965e6deebb35f784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:28.943550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:28.944196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:28.945771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 958/1891] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:28 2022] DELETE /identity/v3/projects/d89296c0c8f74d37965e6deebb35f784 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:28.949987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:28.951917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:28.975285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:28.975950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:28.976231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:28.976561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] PATH_INFO: `/v3/users/1e22b03199144741859f0149382d4e67` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:28.977901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e22b03199144741859f0149382d4e67)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:28.980481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:29.023753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bddb81-e36a-46ab-95bc-8010f195c27d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:29.029519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:29.030377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:29.030651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:29.030921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] PATH_INFO: `/v3/users/e47046f1bc8d43f3b15860144049b2a4` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:29.032092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47046f1bc8d43f3b15860144049b2a4)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:29.034241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:29.250856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e22b03199144741859f0149382d4e67'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.251566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e22b03199144741859f0149382d4e67'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.282716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47046f1bc8d43f3b15860144049b2a4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.283783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47046f1bc8d43f3b15860144049b2a4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.297551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47046f1bc8d43f3b15860144049b2a4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.349067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:29.377577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e22b03199144741859f0149382d4e67'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.388481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e22b03199144741859f0149382d4e67'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.402093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e03c8f5e-b6a8-45e0-a644-e6d7cf6c2e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e22b03199144741859f0149382d4e67'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.408542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 934/1892] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:28 2022] DELETE /identity/v3/users/1e22b03199144741859f0149382d4e67 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:29.414283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:29.414560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:29.414827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:29.435680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:29.436935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:29.481608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47046f1bc8d43f3b15860144049b2a4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.484232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47046f1bc8d43f3b15860144049b2a4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:29.507946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 959/1893] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:28 2022] DELETE /identity/v3/users/e47046f1bc8d43f3b15860144049b2a4 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:29.521587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 935/1894] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:29.537627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:29.540245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:29.619394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27407-1c06-4214-b3e1-37c00f652e7f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:29.624134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:29.624851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:29.625118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:29.625544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:29.703211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:29.704710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:29.718500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 960/1895] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:29.935383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:30.029676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:30.091387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.092878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.105465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12569f7-629b-4e9b-9a5c-349879d7b45d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.110386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.111254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.111559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.111863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] PATH_INFO: `/v3/projects/91bc8faec2984d3398b98cebdd0b63a6` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.113330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91bc8faec2984d3398b98cebdd0b63a6)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.115568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.286590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.290395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.301989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91bc8faec2984d3398b98cebdd0b63a6', 'request_id': 'req-561a741d-8074-471f-b9cf-452bd0768643'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.305469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99297647-b8ac-4a2a-a79b-012ccb8dd706 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.309265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91bc8faec2984d3398b98cebdd0b63a6', 'request_id': 'req-561a741d-8074-471f-b9cf-452bd0768643'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.369061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:30.369268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.369995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.370762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.371123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] PATH_INFO: `/v3/projects/0b6ba016de694fe783a9b6235eec944d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.375081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6ba016de694fe783a9b6235eec944d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.375081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.395495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91bc8faec2984d3398b98cebdd0b63a6', 'request_id': 'req-561a741d-8074-471f-b9cf-452bd0768643'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.409988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] The token cache is being invalidate because project 91bc8faec2984d3398b98cebdd0b63a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:30.410374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.411117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-561a741d-8074-471f-b9cf-452bd0768643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.412782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 936/1896] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:30 2022] DELETE /identity/v3/projects/91bc8faec2984d3398b98cebdd0b63a6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:30.417703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55950-1370-4554-87f3-f4a4d84dbbcf None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.418141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55950-1370-4554-87f3-f4a4d84dbbcf None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.418599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b55950-1370-4554-87f3-f4a4d84dbbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.441971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e4b55950-1370-4554-87f3-f4a4d84dbbcf None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:30.443359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e4b55950-1370-4554-87f3-f4a4d84dbbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.454229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 937/1897] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:30.462528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.463386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.463386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.483878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:30.485117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.497664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 938/1898] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:30.498668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.508260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2540385-ff4f-41fa-ac92-ba72c973d0f0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.513726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.514551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.514664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.514896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.515546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:30.517463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.517579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.533383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 939/1899] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:25:30.534271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6ba016de694fe783a9b6235eec944d', 'request_id': 'req-b473bf23-dd50-41e0-8a3c-a74d92179eb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.537699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.541321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6ba016de694fe783a9b6235eec944d', 'request_id': 'req-b473bf23-dd50-41e0-8a3c-a74d92179eb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.548581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9c74-a33b-43e8-91c5-710ab3732dbe admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.572279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:30.621268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.622076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.622354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.622624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.623322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.625379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.632427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6ba016de694fe783a9b6235eec944d', 'request_id': 'req-b473bf23-dd50-41e0-8a3c-a74d92179eb3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.646733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] The token cache is being invalidate because project 0b6ba016de694fe783a9b6235eec944d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:30.647114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.647860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:30.649474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 961/1900] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:30 2022] DELETE /identity/v3/projects/0b6ba016de694fe783a9b6235eec944d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:30.654075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.655656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.661991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:30.674433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473bf23-dd50-41e0-8a3c-a74d92179eb3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.679597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.680250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.680525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.680798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] PATH_INFO: `/v3/users/0aa7564ee6db415a97f741a1b35ca936` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.682382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa7564ee6db415a97f741a1b35ca936)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.690566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.739648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 940/1901] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:30 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:30.744058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.747532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:30.831872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3deaebf-a1df-4cbc-a3d8-8a08b1e268c9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.836710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.837331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.837613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.837916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:30.838558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:30.840723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:30.972994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 941/1902] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:30 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:30.977680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:30.991555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17022cc2-58cf-452e-8b32-d552cacba750 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:30.998612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:30.999355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:30.999670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:30.999955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.000718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.002221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.015039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa7564ee6db415a97f741a1b35ca936'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:31.019420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa7564ee6db415a97f741a1b35ca936'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:31.024076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 942/1903] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:31.028868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.032542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:31.033475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa7564ee6db415a97f741a1b35ca936'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:31.063285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e7884-c7c6-47c5-aad7-24b2b1f855b2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.068320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.068950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.069226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.069496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.070175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.071793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.090618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 943/1904] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:31.095060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.107038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb1799-3d7e-4de8-b6a4-81c125670f02 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.111933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.112624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.112909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.113202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] PATH_INFO: `/v3/projects/2b1917b1128d4c7596fc388e0cfe9f26/users/d55172ca10004003a83c71c349bf49e9/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.122558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa7564ee6db415a97f741a1b35ca936'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:31.123206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-cdee8390-b661-49d8-9c56-b84611ab051e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa7564ee6db415a97f741a1b35ca936'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:31.130179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1917b1128d4c7596fc388e0cfe9f26, user_id=d55172ca10004003a83c71c349bf49e9, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.133166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.146936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 962/1905] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:25:30 2022] DELETE /identity/v3/users/0aa7564ee6db415a97f741a1b35ca936 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:31.227175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 944/1906] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:31 2022] PUT /identity/v3/projects/2b1917b1128d4c7596fc388e0cfe9f26/users/d55172ca10004003a83c71c349bf49e9/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:31.240096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.240366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.240639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.270647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] MFA Rules not processed for user `d55172ca10004003a83c71c349bf49e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:31.271881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:31.342986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 963/1907] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:31.347814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.382549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:31.439104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2350-94df-46b4-86d0-18f431f35538 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.445672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.446399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.446741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.446932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.447647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.451222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.662255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 945/1908] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:31 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:31.664014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.669617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560412-e1ea-4a32-aa7b-c7e64b5641f2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.715948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:31.754677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.754677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.754677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.754677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.755020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.755020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.868372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.871040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:31.884292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf240de0-1270-4c7b-8015-4ee652088cf6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.894655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.895450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.895818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.896166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] PATH_INFO: `/v3/projects/76344879f078409ab575b106a6e8a084` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.897572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76344879f078409ab575b106a6e8a084)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.899632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.930550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 964/1909] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:31 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:31.935018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.957493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090ae3-bee8-465f-93b1-2573745bda07 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.957493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.957493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.957493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.958319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.958532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.960199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:31.973399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 965/1910] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:31.977828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:31.989329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bdc97-5b6d-44ec-bc24-26be4ce08618 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:31.994289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:31.994911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:31.995254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:31.996069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:31.996719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:31.998356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:32.009898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 966/1911] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:32.014202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:32.024645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99dda1-e8fa-445b-8ceb-7a5bdf603260 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:32.031954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:32.032698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.032976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.033253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] PATH_INFO: `/v3/projects/c49a7ab74ff34d53ad47f24d1ebdd66c/users/43f9b04e4e574aac9e6cfa251338def7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.059315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49a7ab74ff34d53ad47f24d1ebdd66c, user_id=43f9b04e4e574aac9e6cfa251338def7, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:32.062482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38734a-7a95-4685-8102-9f2987195ec9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:32.559708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 967/1912] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:32 2022] PUT /identity/v3/projects/c49a7ab74ff34d53ad47f24d1ebdd66c/users/43f9b04e4e574aac9e6cfa251338def7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:32.570528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296df80-3a01-4a86-9510-39657b338836 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.570911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296df80-3a01-4a86-9510-39657b338836 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.571266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296df80-3a01-4a86-9510-39657b338836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.573475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76344879f078409ab575b106a6e8a084', 'request_id': 'req-ad4e3fb0-f88c-4654-81fd-9e75f7814916'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:32.581443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76344879f078409ab575b106a6e8a084', 'request_id': 'req-ad4e3fb0-f88c-4654-81fd-9e75f7814916'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:32.599873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0296df80-3a01-4a86-9510-39657b338836 None None] MFA Rules not processed for user `43f9b04e4e574aac9e6cfa251338def7`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:32.604717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0296df80-3a01-4a86-9510-39657b338836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:32.655340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76344879f078409ab575b106a6e8a084', 'request_id': 'req-ad4e3fb0-f88c-4654-81fd-9e75f7814916'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:32.666728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] The token cache is being invalidate because project 76344879f078409ab575b106a6e8a084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:25:32.667022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:32.667705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ad4e3fb0-f88c-4654-81fd-9e75f7814916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:25:32.669883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 946/1913] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:25:31 2022] DELETE /identity/v3/projects/76344879f078409ab575b106a6e8a084 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:32.674623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66e841-69db-42f9-a1dd-837dc53f0c1d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.674911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66e841-69db-42f9-a1dd-837dc53f0c1d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.675188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66e841-69db-42f9-a1dd-837dc53f0c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.678073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 968/1914] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:32.682299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296df80-3a01-4a86-9510-39657b338836 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:32.690089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:32.698362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ab66e841-69db-42f9-a1dd-837dc53f0c1d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:32.699616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ab66e841-69db-42f9-a1dd-837dc53f0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:32.767336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296df80-3a01-4a86-9510-39657b338836 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:32.780789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:32.781932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.782419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.782831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.783881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:32.812635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:32.815598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 947/1915] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:25:32.819162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.819903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.820414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.903405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:32.915418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:32.935407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 948/1916] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:32.940385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:32.950775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8ea1c-e5f5-47bc-9141-ff5dd654be26 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:32.955971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:32.958535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:32.958535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:32.958535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:32.958535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:25:32.958535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:32.960724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:32.966343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 969/1917] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/projects => generated 377 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:32.967317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:32.971077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:32.979354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 949/1918] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:25:32.981128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:32.998510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f4c01-2fa7-4846-8388-d9021144840a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.071567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.072468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.072820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.073217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.074001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.076766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.094027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf31f4-59a6-4de0-842d-51d01bb8c401 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.101225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.103491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.104214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.104754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.107184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.117316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.236810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 950/1919] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:33.241618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.263788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 970/1920] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:32 2022] POST /identity/v3/users => generated 295 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:33.266276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.285235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499449c-d414-4090-a60a-227bc8728282 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.315971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:33.347900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb093a4-f7ab-43d2-8ace-aff2c80eb172 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.355247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.357001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.357540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.358005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.358998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.361758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.365462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.366127 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.366429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.366694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.367374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.369752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.385806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 971/1921] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:33.389654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.400365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ada3-78f9-4c2c-869e-dbdd31a7bf52 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.406020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.406758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.407038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.407311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] PATH_INFO: `/v3/projects/18955f203fe741d1b05d792ec669b1d7/users/e889ccb2c9c04f189b5b86073c68cc3e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.427305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18955f203fe741d1b05d792ec669b1d7, user_id=e889ccb2c9c04f189b5b86073c68cc3e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.430079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.504352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 951/1922] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:33 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:25:33.511010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.523304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262770a0-35da-43f4-96b9-f7494830ea57 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.533124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 972/1923] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:33 2022] PUT /identity/v3/projects/18955f203fe741d1b05d792ec669b1d7/users/e889ccb2c9c04f189b5b86073c68cc3e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:33.537117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.592483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.593118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.593118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.593303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.593926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.595601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.602188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:33.607015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 952/1924] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:33.608470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd2cf9-daaf-46ff-972c-650fe5a2c872 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.611465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.613297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.613966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.614173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.614423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.616463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.617161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.623062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6106b4-3be5-4c4c-bb38-ca436dc8c794 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.629931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 973/1925] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:33.630114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.633238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.633238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.633238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.633238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.633552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:33.634277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.635542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.647154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b2f06-1361-487c-a2cd-6216b379fa38 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.650559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 953/1926] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:33.655266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.655771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.656475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.656768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.657022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] PATH_INFO: `/v3/projects/18955f203fe741d1b05d792ec669b1d7/users/e889ccb2c9c04f189b5b86073c68cc3e/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.658934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18955f203fe741d1b05d792ec669b1d7, user_id=e889ccb2c9c04f189b5b86073c68cc3e, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.661806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2873dc6-b88b-4633-92db-dc7c1c9a71f6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.665852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfbbe2-5393-4f82-a241-d63edff94421 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.671050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.671734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.672012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.672279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] PATH_INFO: `/v3/projects/a96eea4543f94dd18842d831b931da33/users/46c6a073213343c89c55e965e9828684/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.673112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 974/1927] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:33 2022] PUT /identity/v3/projects/18955f203fe741d1b05d792ec669b1d7/users/e889ccb2c9c04f189b5b86073c68cc3e/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:33.683452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c766-bd95-4ef9-a976-423b9576da11 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.683738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c766-bd95-4ef9-a976-423b9576da11 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.684028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556c766-bd95-4ef9-a976-423b9576da11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.694717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96eea4543f94dd18842d831b931da33, user_id=46c6a073213343c89c55e965e9828684, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.697492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.713191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-1556c766-bd95-4ef9-a976-423b9576da11 None None] MFA Rules not processed for user `e889ccb2c9c04f189b5b86073c68cc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:33.714468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1556c766-bd95-4ef9-a976-423b9576da11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:33.789378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 975/1928] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:33.795677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 954/1929] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:33 2022] PUT /identity/v3/projects/a96eea4543f94dd18842d831b931da33/users/46c6a073213343c89c55e965e9828684/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:33.801058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.801376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.801877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.802599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.804323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:33.824333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:33.826613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:33.873942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b2883-f5f0-4f19-9836-c10496a92d1c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:33.878295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:33.879464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:33.879726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:33.880009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:33.881940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:33.902098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 976/1930] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:33.909498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:33.911112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:33.928842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:33.966554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:33.966554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f22231-9694-4a6c-b46d-0dac0a38bd4b service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:33.975845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 955/1931] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:33.995841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:34.005677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dcfd8-4b86-4e8e-8d45-70f585a16a8c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:34.013133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:34.013825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:34.014124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:34.014387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:34.029331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:34.031313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:34.053008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 977/1932] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:25:34.590490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:34.658985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:35.470166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:35.876990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:35.877653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:35.877792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:35.942189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] MFA Rules not processed for user `46c6a073213343c89c55e965e9828684`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:35.948318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:35.984637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:36.121471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 956/1933] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:36.124161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.147913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccb5a4-91e7-4950-935d-ca9e5d537a7d service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.156640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.157503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.157981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.158653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.159454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.168412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.333188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 978/1934] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:25:36 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:36.334365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.347988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278c059-1583-4f7e-b31a-11d3825ac292 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.427971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.459792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.459792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.459792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.459792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.459792 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.632262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 957/1935] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:25:36 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:25:36.635598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.644933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24242f91-59b5-472c-a0bb-83c62a306e00 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.652821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.660180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.660180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.660180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.660180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.660180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.677434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 979/1936] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:25:36.680618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.694449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc61e2-5763-411d-8140-4c94555f0036 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.700245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.701460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.702101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.702101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] PATH_INFO: `/v3/projects/9925a33340a24bc3b9988f3e6d2879e9/users/950cd0f1ec58470692911732a747a227/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.725637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9925a33340a24bc3b9988f3e6d2879e9, user_id=950cd0f1ec58470692911732a747a227, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.732365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.809471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 958/1937] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:36 2022] PUT /identity/v3/projects/9925a33340a24bc3b9988f3e6d2879e9/users/950cd0f1ec58470692911732a747a227/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:25:36.812827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.824784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f073-aa7e-4665-acb5-a1c91e27336a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.826539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.828906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:36.904369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a8142-b659-4580-8483-1ee444a29574 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.909380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.910139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.910415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.910739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.910878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.911402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.911872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.912325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.913010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.913157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.915669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:36.925439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 980/1938] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:25:36.930339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:36.944348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aae22-9cbb-479c-ae17-52aaf203226f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:36.952724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:36.953563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:36.953930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:36.954212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] PATH_INFO: `/v3/projects/9925a33340a24bc3b9988f3e6d2879e9/users/950cd0f1ec58470692911732a747a227/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:36.956389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9925a33340a24bc3b9988f3e6d2879e9, user_id=950cd0f1ec58470692911732a747a227, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:36.959351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deac614-69db-4c8d-988a-2e9367a1c6be admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:36.976441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 981/1939] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:25:36 2022] PUT /identity/v3/projects/9925a33340a24bc3b9988f3e6d2879e9/users/950cd0f1ec58470692911732a747a227/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:25:37.029216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:37.031045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:37.046237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 959/1940] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:37.127407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:37.209608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:38.236060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:38.389089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:38.732803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:38.733087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:38.733355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:38.764844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] MFA Rules not processed for user `950cd0f1ec58470692911732a747a227`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:25:38.766126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:25:38.860155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 982/1941] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:25:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:25:38.901538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:38.910084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a26e7-9752-4520-a712-aac0f376eecd service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:38.962108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:38.962699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:38.962976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:38.963248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:39.004212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:39.024352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:39.025879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:39.036041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 960/1942] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:40.555525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:40.565329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8871501-a964-47cf-b801-7d97ab15a8a9 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:40.568876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:40.569518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:40.569820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:40.570089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:40.632714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:40.634238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:40.644105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 983/1943] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:25:49.281121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:50.639845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:25:54.679432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:25:54.692847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e957c24-41f6-4298-bef9-c059a093312c service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:25:54.698933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:25:54.699602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:25:54.699874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:25:54.700137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:25:54.710623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:25:54.712040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0069444-3129-4fe9-8ec7-cdcaddbdcafb service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:25:54.723165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 961/1944] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:04.491323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:04.582359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f4996-7716-4175-b0b5-8b7dd1639c9e service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:04.587698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:04.588505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:04.588787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:04.589066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] PATH_INFO: `/v3/users/d55172ca10004003a83c71c349bf49e9` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:04.590259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55172ca10004003a83c71c349bf49e9)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:04.592228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:04.717385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:04.886607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55172ca10004003a83c71c349bf49e9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:04.887177 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55172ca10004003a83c71c349bf49e9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:04.894889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55172ca10004003a83c71c349bf49e9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:04.997706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55172ca10004003a83c71c349bf49e9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:04.998367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55172ca10004003a83c71c349bf49e9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:05.018129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 984/1945] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:04 2022] DELETE /identity/v3/users/d55172ca10004003a83c71c349bf49e9 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:05.023175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:05.023467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:05.023735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:05.043646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:05.044966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:05.105683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 962/1946] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:05.115837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:05.117374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:05.177002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683bcd6d-de8b-4946-b293-afc43f4947a1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:05.180752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:05.181369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:05.181645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:05.182027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:05.191739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:05.193253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:05.204071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 985/1947] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:26:05.669893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:05.673898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:05.686542 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfcd5a-8138-45c6-8514-929407d029dd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:05.694238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:05.695224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:05.695693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:05.696155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] PATH_INFO: `/v3/projects/2b1917b1128d4c7596fc388e0cfe9f26` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:05.698173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1917b1128d4c7596fc388e0cfe9f26)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:05.701702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:05.855278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1917b1128d4c7596fc388e0cfe9f26', 'request_id': 'req-39ac1b08-41d8-4256-915a-c30550cb6590'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:05.861582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1917b1128d4c7596fc388e0cfe9f26', 'request_id': 'req-39ac1b08-41d8-4256-915a-c30550cb6590'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.049376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1917b1128d4c7596fc388e0cfe9f26', 'request_id': 'req-39ac1b08-41d8-4256-915a-c30550cb6590'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.061238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] The token cache is being invalidate because project 2b1917b1128d4c7596fc388e0cfe9f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:06.063393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.063393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.064398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 963/1948] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:05 2022] DELETE /identity/v3/projects/2b1917b1128d4c7596fc388e0cfe9f26 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:06.068532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:06.079121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fc23f-d46e-4f66-ac91-8b73732e8323 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:06.098032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:06.137099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:06.137833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:06.138033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:06.138309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] PATH_INFO: `/v3/users/43f9b04e4e574aac9e6cfa251338def7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:06.139523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43f9b04e4e574aac9e6cfa251338def7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:06.142113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:06.399135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43f9b04e4e574aac9e6cfa251338def7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.400223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43f9b04e4e574aac9e6cfa251338def7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.408014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43f9b04e4e574aac9e6cfa251338def7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.501703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43f9b04e4e574aac9e6cfa251338def7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.502983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43f9b04e4e574aac9e6cfa251338def7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:06.526900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 986/1949] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:06 2022] DELETE /identity/v3/users/43f9b04e4e574aac9e6cfa251338def7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:07.040228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:07.041665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:07.056045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac1b08-41d8-4256-915a-c30550cb6590 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:07.117408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:07.118138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:07.118435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:07.118710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] PATH_INFO: `/v3/projects/c49a7ab74ff34d53ad47f24d1ebdd66c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:07.120135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49a7ab74ff34d53ad47f24d1ebdd66c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:07.122272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:07.473940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:07.490448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49a7ab74ff34d53ad47f24d1ebdd66c', 'request_id': 'req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.496687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49a7ab74ff34d53ad47f24d1ebdd66c', 'request_id': 'req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.601650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49a7ab74ff34d53ad47f24d1ebdd66c', 'request_id': 'req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.616853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] The token cache is being invalidate because project c49a7ab74ff34d53ad47f24d1ebdd66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:07.617148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.617901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.622288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 964/1950] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:07 2022] DELETE /identity/v3/projects/c49a7ab74ff34d53ad47f24d1ebdd66c => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:07.623477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:07.625484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:07.638625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4381a66-83ca-44f1-baa2-03b42e35a03d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:07.710694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:07.711361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:07.711640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:07.711911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] PATH_INFO: `/v3/users/e889ccb2c9c04f189b5b86073c68cc3e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:07.713067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e889ccb2c9c04f189b5b86073c68cc3e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:07.715571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:07.929475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:07.932041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e889ccb2c9c04f189b5b86073c68cc3e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.932702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e889ccb2c9c04f189b5b86073c68cc3e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:07.939786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e889ccb2c9c04f189b5b86073c68cc3e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.134453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e889ccb2c9c04f189b5b86073c68cc3e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.135172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e889ccb2c9c04f189b5b86073c68cc3e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.159574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 987/1951] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:07 2022] DELETE /identity/v3/users/e889ccb2c9c04f189b5b86073c68cc3e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:08.474099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:08.475415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:08.488583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0778f-d00c-4e3f-9ab9-00e11e58dcf7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:08.549404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:08.550128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:08.550340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:08.550615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] PATH_INFO: `/v3/users/d33b063bd44043a7a2f12be05e48b48b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:08.552141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33b063bd44043a7a2f12be05e48b48b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:08.554249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:08.638798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:08.640563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:08.677675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d146b2d-e537-48e8-b646-433ffabc4695 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:08.684846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:08.685573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:08.686019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:08.691036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] PATH_INFO: `/v3/projects/18955f203fe741d1b05d792ec669b1d7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:08.691036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18955f203fe741d1b05d792ec669b1d7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:08.692625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:08.783050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33b063bd44043a7a2f12be05e48b48b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.788550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33b063bd44043a7a2f12be05e48b48b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.828136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18955f203fe741d1b05d792ec669b1d7', 'request_id': 'req-1d25ad99-4ea4-4f4d-9c30-164110569835'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.839575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18955f203fe741d1b05d792ec669b1d7', 'request_id': 'req-1d25ad99-4ea4-4f4d-9c30-164110569835'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.916579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33b063bd44043a7a2f12be05e48b48b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.924640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33b063bd44043a7a2f12be05e48b48b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.936774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-063bf016-ba95-4f21-9890-f5cdfff87436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33b063bd44043a7a2f12be05e48b48b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.943597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 965/1952] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:08 2022] DELETE /identity/v3/users/d33b063bd44043a7a2f12be05e48b48b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:08.947815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:08.948093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:08.948360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:08.968117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:08.969950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:08.980701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18955f203fe741d1b05d792ec669b1d7', 'request_id': 'req-1d25ad99-4ea4-4f4d-9c30-164110569835'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.993310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] The token cache is being invalidate because project 18955f203fe741d1b05d792ec669b1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:08.993601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.994350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1d25ad99-4ea4-4f4d-9c30-164110569835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:08.995991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 988/1953] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:08 2022] DELETE /identity/v3/projects/18955f203fe741d1b05d792ec669b1d7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:09.000789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cc2d2-e3a7-4f15-bed6-347ed52d5b30 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.001074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cc2d2-e3a7-4f15-bed6-347ed52d5b30 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.001346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cc2d2-e3a7-4f15-bed6-347ed52d5b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.004081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:09.021569 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a02cc2d2-e3a7-4f15-bed6-347ed52d5b30 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:09.022868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a02cc2d2-e3a7-4f15-bed6-347ed52d5b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:09.035841 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 966/1954] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:09.045960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.047964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:09.084917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 989/1955] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:09.092871 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.093340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.093622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.107578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4a414-9bb0-4e06-90e7-7524912553f2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.112462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.113079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.113358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.113630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.119655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:09.120891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:09.123431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.124875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.129186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 990/1956] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:09.133659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.138355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 967/1957] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:26:09.142808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4916ce-988a-48d2-887e-815e6933124b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.147728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.149672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.149672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.149672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.149672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:09.150045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.151480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.165938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 991/1958] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:09.170034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.183925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac892c1-5f0b-49a7-905f-d261986d8708 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.201473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.201473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.201473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.201473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.202031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.202031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.325830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 968/1959] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:09 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:09.330807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.342086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb218009-f7dc-4bbd-b13a-cd70638423a1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.364233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:09.408681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.410622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.411026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.411738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.412397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.424948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.528735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 992/1960] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:09 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:09.534741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.545171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58e4a4-bc49-4853-86d1-b90ed068ef96 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.549984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.555556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.555556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.555556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.555556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.555556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.564605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 969/1961] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:09.568713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.579756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a26f44-71c4-4e56-b66d-9bded91c2f0f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.589596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.589596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.589596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.589596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.589988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.589988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.600030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 993/1962] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:09.604538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.616441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadc1e-6051-47b7-b7dc-2b6c057d9df1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.621239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.621960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.622242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.622521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] PATH_INFO: `/v3/projects/b10dca5c2ea7435690051684c1113800/users/55dfe3112e6f4698bf9a821c217e3d59/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.639959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10dca5c2ea7435690051684c1113800, user_id=55dfe3112e6f4698bf9a821c217e3d59, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.643565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb700bd-e649-486b-9914-116db9eee88a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.692897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.694611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:09.713760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb6b9f-fc4a-4850-af03-db5c6f9a6358 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:09.722904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:09.722904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.722904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.722904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] PATH_INFO: `/v3/projects/377a2ea603f6470d85a36d9384d9e92e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.723240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] RBAC: Authorizing `identity:delete_project(project_id=377a2ea603f6470d85a36d9384d9e92e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:09.723729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:09.776675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 970/1963] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:09 2022] PUT /identity/v3/projects/b10dca5c2ea7435690051684c1113800/users/55dfe3112e6f4698bf9a821c217e3d59/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:09.787945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:09.793466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:09.793466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:09.818861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] MFA Rules not processed for user `55dfe3112e6f4698bf9a821c217e3d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:09.820110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:09.885827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377a2ea603f6470d85a36d9384d9e92e', 'request_id': 'req-4173524e-9da5-485f-b68c-9656d2e6c64a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:09.887080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 971/1964] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:09.891431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:09.893509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377a2ea603f6470d85a36d9384d9e92e', 'request_id': 'req-4173524e-9da5-485f-b68c-9656d2e6c64a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:09.901689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:09.955148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201584b-24ea-45e3-890b-1fc67531e1cc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.012622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.013236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.013522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.013825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.014459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.016607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.036226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377a2ea603f6470d85a36d9384d9e92e', 'request_id': 'req-4173524e-9da5-485f-b68c-9656d2e6c64a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.048399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] The token cache is being invalidate because project 377a2ea603f6470d85a36d9384d9e92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:10.048684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.049379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.050974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 994/1965] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:09 2022] DELETE /identity/v3/projects/377a2ea603f6470d85a36d9384d9e92e => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:10.054792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:10.056099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:10.068834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173524e-9da5-485f-b68c-9656d2e6c64a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.073499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.074144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.074415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.074682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] PATH_INFO: `/v3/users/fb3ac2f7137e46fa831023ac1883cf98` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.075794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3ac2f7137e46fa831023ac1883cf98)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.077701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.215863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 972/1966] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:09 2022] POST /identity/v3/projects => generated 385 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:10.220616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:10.222334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:10.292696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4b88-2fdd-4028-a620-cfb84abd8562 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.297469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.298152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.298435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.298725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.299400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.301335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.445139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3ac2f7137e46fa831023ac1883cf98'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.445795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3ac2f7137e46fa831023ac1883cf98'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.453265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3ac2f7137e46fa831023ac1883cf98'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.491189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 973/1967] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:10 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:10.494941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:10.497006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:10.512251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6deb8-c461-4d01-b4a6-926b61d616a1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.520604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.525545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.525545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.525545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.525545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.525545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.539425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 974/1968] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:26:10.543190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:10.553469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180d10-279f-4085-a3aa-6bca0970c98c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.558279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.558886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.559166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.559443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.560115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.561842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.572654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 975/1969] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:26:10.576903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:10.587243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafa5d7-ebc1-49af-90a2-549c2bd385fe admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:10.592014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:10.592666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.592960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.593219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] PATH_INFO: `/v3/projects/67bedceffddb46a4887f5b2b53cd0f02/users/00c27ad41e774886a8e7626ce481b18d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.608290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bedceffddb46a4887f5b2b53cd0f02, user_id=00c27ad41e774886a8e7626ce481b18d, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:10.611372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:10.695597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3ac2f7137e46fa831023ac1883cf98'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.696267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-69c7f93b-9aef-4275-9491-259df941eef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3ac2f7137e46fa831023ac1883cf98'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:10.710565 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:10.715683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 995/1970] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:10 2022] DELETE /identity/v3/users/fb3ac2f7137e46fa831023ac1883cf98 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:10.730519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:10.791220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 976/1971] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:10 2022] PUT /identity/v3/projects/67bedceffddb46a4887f5b2b53cd0f02/users/00c27ad41e774886a8e7626ce481b18d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:10.802872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:10.803192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:10.803438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:10.852955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] MFA Rules not processed for user `00c27ad41e774886a8e7626ce481b18d`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:10.854338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:10.927829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 996/1972] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:10.932398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.006254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517992b-dbe9-4789-ab43-5735ca4a19e6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.012068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.012484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.012781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.013181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.013860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.015779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.154283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 977/1973] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:10 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:11.158591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.169706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e3cef-e53e-4e8b-b2f0-7bce973bfc4b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.180931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.189294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.258436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.260367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.261128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.261429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.262304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.265152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.278454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e9356-d85d-4c1b-80f6-e15b383cb1cc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.284062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.284715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.284995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.285269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] PATH_INFO: `/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.286497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a7e6d99c5d483aab8e56f9e7915aaa)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.288544 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.387930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 997/1974] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:11.392977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.404584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369089c-5e86-4933-8c4a-c783f7e4d4e9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.411870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a7e6d99c5d483aab8e56f9e7915aaa', 'request_id': 'req-7a481c53-0f50-4676-ba90-62ef8167dbe4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:11.419683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a7e6d99c5d483aab8e56f9e7915aaa', 'request_id': 'req-7a481c53-0f50-4676-ba90-62ef8167dbe4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:11.462468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b568216-11f2-493e-b997-71283001852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.463080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568216-11f2-493e-b997-71283001852a admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.463352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568216-11f2-493e-b997-71283001852a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.463621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b568216-11f2-493e-b997-71283001852a admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.464263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b568216-11f2-493e-b997-71283001852a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.465893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b568216-11f2-493e-b997-71283001852a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.478584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 998/1975] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:26:11.481136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b568216-11f2-493e-b997-71283001852a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.491185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b568216-11f2-493e-b997-71283001852a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.495769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.496408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.496683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.496955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] PATH_INFO: `/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac/users/44d369c414904bb9919749126e6ed197/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.511744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e711a2b5b64e7a8384e50c83e944ac, user_id=44d369c414904bb9919749126e6ed197, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.514429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.576379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a7e6d99c5d483aab8e56f9e7915aaa', 'request_id': 'req-7a481c53-0f50-4676-ba90-62ef8167dbe4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:11.588627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:11.593365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] The token cache is being invalidate because project d5a7e6d99c5d483aab8e56f9e7915aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:11.593662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:11.594438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a481c53-0f50-4676-ba90-62ef8167dbe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:11.598996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 978/1976] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:11 2022] DELETE /identity/v3/projects/d5a7e6d99c5d483aab8e56f9e7915aaa => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:11.606285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc461c-9e9c-43bb-91fa-b628f4cf0a31 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.606779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc461c-9e9c-43bb-91fa-b628f4cf0a31 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.607383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc461c-9e9c-43bb-91fa-b628f4cf0a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.623295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:11.637650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a2cc461c-9e9c-43bb-91fa-b628f4cf0a31 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:11.644441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a2cc461c-9e9c-43bb-91fa-b628f4cf0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.652873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 979/1977] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:11.663048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.663438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.663902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.664790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 999/1978] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:11 2022] PUT /identity/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac/users/44d369c414904bb9919749126e6ed197/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:11.675158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.677087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.693448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:11.695939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.762706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d6390-0e46-4e28-be29-efbd60e6ec46 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.765484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 980/1979] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:11.768125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.768734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.769040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.769403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.770086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.770428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.771690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.779161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d17e0-4be6-4059-9160-abbf928f7dd9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.782786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1000/1980] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:26:11.785457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.786100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.788137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.788137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.788137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:11.788137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.789141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.791102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.804698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 981/1981] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:26:11.804898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5afed-3345-4325-9e1b-4095d81f9f99 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.805661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.806785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.807443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.807724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.807999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] PATH_INFO: `/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac/users/44d369c414904bb9919749126e6ed197/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.809978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e711a2b5b64e7a8384e50c83e944ac, user_id=44d369c414904bb9919749126e6ed197, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.812654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93a6c7-87ef-4998-8cae-47e22486dacd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.815429 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3097-d5df-4de3-bb7d-0f6734a9103e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:11.820171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:11.821237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.821237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.821405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.821911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:11.823479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1001/1982] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:11 2022] PUT /identity/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac/users/44d369c414904bb9919749126e6ed197/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:11.823806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:11.834913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:11.835326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:11.835605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:11.867230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] MFA Rules not processed for user `44d369c414904bb9919749126e6ed197`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:11.868429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.941637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1002/1983] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:11.956220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:11.957630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:11.973371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 982/1984] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:11.976816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.025226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06decf9-3e36-43f9-a5db-026a131b170d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.050720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e363ca-1aab-4a61-a6cf-e521fd100528 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.055876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.056551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.056826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.057102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.057769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.059698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.082145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.089815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.089815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.089815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.089815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:12.097524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:12.150021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.151516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.161525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1003/1985] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:26:12.206662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 983/1986] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:11 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:12.209590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.221515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6909ab1-b775-4efe-ace5-6c4e0047b224 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.226662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.227287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.227567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.227841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.228488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.230605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.241508 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1004/1987] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:12.245551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.256405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe95ee-71e2-4a06-93ec-9efec25aa419 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.261604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.262264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.262680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.262958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.263612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.265274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02779e32-a53b-472a-8275-747d5a4ce953 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.280996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 984/1988] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:12.284898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.295034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8aa421-d21b-4a2a-9603-02f4d2abb748 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.300659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.301331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.301608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.301919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] PATH_INFO: `/v3/projects/d45a494827004cb5a392662d8a6a727a/users/717423b700ad475792be81c05335560f/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.325372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45a494827004cb5a392662d8a6a727a, user_id=717423b700ad475792be81c05335560f, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.331054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.440821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1005/1989] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:12 2022] PUT /identity/v3/projects/d45a494827004cb5a392662d8a6a727a/users/717423b700ad475792be81c05335560f/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:12.448244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.448539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.448823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.483023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] MFA Rules not processed for user `717423b700ad475792be81c05335560f`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:12.484237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:12.529466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:12.568989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 985/1990] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:12.573789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.651309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cf656-bd8e-4733-8208-9c206949bc35 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.656710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.657820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.658022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.658287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.659453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.661458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.790588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1006/1991] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:12 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:12.791763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.803827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f531896-acaa-41f9-9557-2950ded06604 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.872927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.873561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.873875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.874149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.874788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.877988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:12.976166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 986/1992] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:12 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:12.980391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:12.991136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6821-7de6-495d-860c-a279b06e14fe admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:12.995746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:12.996363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:12.996640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:12.996947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:12.997596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:12.999329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:13.011209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1007/1993] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:13.015754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:13.027324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d53c9-f20b-4a73-9e0e-a7023e9bc193 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:13.032648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:13.033357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:13.033640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:13.036534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] PATH_INFO: `/v3/projects/3606045986774fb4b9cfe0fe587670d0/users/5ad5019fc71f4fdd92e426ecb14ebd9d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:13.051702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3606045986774fb4b9cfe0fe587670d0, user_id=5ad5019fc71f4fdd92e426ecb14ebd9d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:13.054523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:13.171395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 987/1994] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:13 2022] PUT /identity/v3/projects/3606045986774fb4b9cfe0fe587670d0/users/5ad5019fc71f4fdd92e426ecb14ebd9d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:13.186087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:13.197288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f9393-f991-488d-a666-dddb1e4f2bf2 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:13.201334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:13.261278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:13.261923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:13.262203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:13.262475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:13.263109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:13.264704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40fa37-badf-491f-b79d-d7b88c28d2df admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:13.275193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1008/1995] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:13.279507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:13.290719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cddd63-8222-452f-8fae-08149be972f3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:13.295584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:13.296275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:13.296555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:13.296830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] PATH_INFO: `/v3/projects/3606045986774fb4b9cfe0fe587670d0/users/5ad5019fc71f4fdd92e426ecb14ebd9d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:13.299366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3606045986774fb4b9cfe0fe587670d0, user_id=5ad5019fc71f4fdd92e426ecb14ebd9d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:13.302288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:13.314163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 988/1996] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:13 2022] PUT /identity/v3/projects/3606045986774fb4b9cfe0fe587670d0/users/5ad5019fc71f4fdd92e426ecb14ebd9d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:13.324969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:13.325255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:13.325532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:13.356456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] MFA Rules not processed for user `5ad5019fc71f4fdd92e426ecb14ebd9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:13.357800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:13.431073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1009/1997] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:13.442719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:13.444467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:13.502504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68259ad9-3ea4-49dc-84a5-92c8feb06bf0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:13.506219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:13.506822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:13.507102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:13.507371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:13.541295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:13.565548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:13.566977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:13.575975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 989/1998] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:17.681883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:17.692455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ef511-a62a-400c-8700-175027fe8065 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:17.696056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:17.696653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:17.696934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:17.697214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:17.699184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:17.763869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:17.765282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:17.774915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1010/1999] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:20.422509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:20.432723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04b6d6-0e56-4522-8442-5eaf7eb15d10 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:20.436716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:20.437379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:20.437662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:20.437998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:20.538467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:20.540017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:20.552972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 990/2000] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:24.621109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:24.634491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b1be-68ff-449e-8b26-b1b25cf24201 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:24.639844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:24.640933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:24.641238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:24.641574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:24.652753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:24.654532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb6005-b6cb-4e28-9fe5-0640bf6a8c8a service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:24.667216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1011/2001] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:26:25.679513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:25.681117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:25.749530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3c253-1fb8-4f0e-946f-e719eba9a9df service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:25.754775 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:25.755458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:25.755728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:25.755997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] PATH_INFO: `/v3/users/a927fa5259b64a288acabf43aedc58b5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:25.757106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a927fa5259b64a288acabf43aedc58b5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:25.759072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:26.151380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a927fa5259b64a288acabf43aedc58b5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:26.152077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a927fa5259b64a288acabf43aedc58b5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:26.262293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a927fa5259b64a288acabf43aedc58b5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:26.269897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a927fa5259b64a288acabf43aedc58b5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:26.275232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:26.282412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a927fa5259b64a288acabf43aedc58b5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:26.289225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 991/2002] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:25 2022] DELETE /identity/v3/users/a927fa5259b64a288acabf43aedc58b5 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:26.293757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:26.294329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:26.294329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:26.314280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:26.316003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:26.324321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:26.384487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1012/2003] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:26.398240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:26.402866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:26.460350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a5c44-4d62-40e5-aede-f646410be82d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:26.465030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:26.465923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:26.473392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:26.473392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:26.478370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:26.480042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:26.492235 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 992/2004] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:27.035311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:27.037901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:27.053462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5afbd6-30fb-4f0e-be26-0637feff483f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:27.059685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:27.060335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:27.060611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:27.060923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] PATH_INFO: `/v3/projects/1713adfd6c434b29b739d7d80f50f5ab` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:27.062178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1713adfd6c434b29b739d7d80f50f5ab)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:27.064086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:27.259845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1713adfd6c434b29b739d7d80f50f5ab', 'request_id': 'req-ff954617-66c5-46d2-92c1-ec5ad556872e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.269242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1713adfd6c434b29b739d7d80f50f5ab', 'request_id': 'req-ff954617-66c5-46d2-92c1-ec5ad556872e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.461662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1713adfd6c434b29b739d7d80f50f5ab', 'request_id': 'req-ff954617-66c5-46d2-92c1-ec5ad556872e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.482888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] The token cache is being invalidate because project 1713adfd6c434b29b739d7d80f50f5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:27.482888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.482888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.482888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1013/2005] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:27 2022] DELETE /identity/v3/projects/1713adfd6c434b29b739d7d80f50f5ab => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:27.485330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:27.498398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946682d4-605a-4491-8c8c-3a567bc6a285 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:27.577130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:27.578259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:27.578736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:27.579211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] PATH_INFO: `/v3/users/1a778fbb39104aa6bc560e11cf87186e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:27.580932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a778fbb39104aa6bc560e11cf87186e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:27.583514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:27.937699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a778fbb39104aa6bc560e11cf87186e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:27.938401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a778fbb39104aa6bc560e11cf87186e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:28.045696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a778fbb39104aa6bc560e11cf87186e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:28.069102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a778fbb39104aa6bc560e11cf87186e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:28.098709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a778fbb39104aa6bc560e11cf87186e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:28.107845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 993/2006] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:27 2022] DELETE /identity/v3/users/1a778fbb39104aa6bc560e11cf87186e => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:28.756763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:28.767412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:28.809580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff954617-66c5-46d2-92c1-ec5ad556872e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:28.880769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:28.883601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:28.899385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:28.943092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:28.944031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:28.944467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:28.944895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] PATH_INFO: `/v3/projects/99bc6f550ce1474eac845789a4f35f3c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:28.948269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bc6f550ce1474eac845789a4f35f3c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:28.951544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:28.987418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809996d-fe92-4a22-9294-4e1bc066fcf3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:28.992256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:28.993072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:28.993349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:28.993651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] PATH_INFO: `/v3/users/55dfe3112e6f4698bf9a821c217e3d59` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:28.994862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dfe3112e6f4698bf9a821c217e3d59)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:28.996835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:29.104029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:29.112894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bc6f550ce1474eac845789a4f35f3c', 'request_id': 'req-c6d1a645-5280-4e66-84d2-71653d1f3d3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.124241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bc6f550ce1474eac845789a4f35f3c', 'request_id': 'req-c6d1a645-5280-4e66-84d2-71653d1f3d3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.296998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bc6f550ce1474eac845789a4f35f3c', 'request_id': 'req-c6d1a645-5280-4e66-84d2-71653d1f3d3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.314673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] The token cache is being invalidate because project 99bc6f550ce1474eac845789a4f35f3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:29.315432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.316418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c6d1a645-5280-4e66-84d2-71653d1f3d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.322503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dfe3112e6f4698bf9a821c217e3d59'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.322503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dfe3112e6f4698bf9a821c217e3d59'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.322930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1014/2007] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:28 2022] DELETE /identity/v3/projects/99bc6f550ce1474eac845789a4f35f3c => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:29.324491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ca26-86f1-4f33-b004-a2aa9b55b038 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.324961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ca26-86f1-4f33-b004-a2aa9b55b038 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.325416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ca26-86f1-4f33-b004-a2aa9b55b038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.345282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a8b2ca26-86f1-4f33-b004-a2aa9b55b038 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:29.347115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a8b2ca26-86f1-4f33-b004-a2aa9b55b038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:29.480217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dfe3112e6f4698bf9a821c217e3d59'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.487468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1015/2008] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:29.495000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.495428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.495747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.507559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dfe3112e6f4698bf9a821c217e3d59'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.518400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:29.520030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:29.534248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1016/2009] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:29.535451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:29.547404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0776b-5a0e-4e5e-8c09-7969ec374fee None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:29.553489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:29.554153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.554720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.555054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.555501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dfe3112e6f4698bf9a821c217e3d59'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:29.555941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:29.556295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:29.558173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530fef1-0f46-4a85-a28d-aa4aef342f3a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:29.577900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1017/2010] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:29.578109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 994/2011] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:28 2022] DELETE /identity/v3/users/55dfe3112e6f4698bf9a821c217e3d59 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:29.579592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:29.583921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.584717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.585111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.627852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:29.629801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:29.694703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424f31f-f69a-4a1c-a1fc-8caf71dd8f34 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:29.701367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:29.703126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.703126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.703575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.704763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:29.708236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:29.722556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1018/2012] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:29.734046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:29.735692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:29.842012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226946b3-a3c7-4771-96be-94f9aae2358d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:29.853175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:29.853175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:29.853175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:29.853175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:29.862215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 995/2013] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:29 2022] POST /identity/v3/projects => generated 387 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:29.865256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:29.888403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:29.890548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:29.910866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1019/2014] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:30.008116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ac6ce-03fb-4341-a282-b8934c0064f8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.014732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.015360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.015686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.015960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.016601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.020022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.199625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 996/2015] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:29 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:30.202822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.229495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c71d-2452-4fc4-b62c-4124f3adc0fd service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.235204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.235964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.237580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.237580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.238138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.239671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.252579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1020/2016] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:30.258279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.276065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73e244-345f-416c-ba25-1076fc44f5ac admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.283104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.284121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.284491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.284771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.285435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.287272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed327a-8dbc-4d41-9e2e-c09d459cadce admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.299051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 997/2017] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:30.303449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.313911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213ade6-cadb-4a35-ba9b-69a0639171e9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.319510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.325400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.325400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.325400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] PATH_INFO: `/v3/projects/d06efda7d2894ac684959b8fb7e8f0bd/users/95ac9bae7b534ec28f4d0341296d0f11/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.336946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06efda7d2894ac684959b8fb7e8f0bd, user_id=95ac9bae7b534ec28f4d0341296d0f11, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.339712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.434943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1021/2018] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:30 2022] PUT /identity/v3/projects/d06efda7d2894ac684959b8fb7e8f0bd/users/95ac9bae7b534ec28f4d0341296d0f11/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:30.446428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.448659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.448659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.481604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] MFA Rules not processed for user `95ac9bae7b534ec28f4d0341296d0f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:30.483225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:30.558913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 998/2019] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:30.566281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.630318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.631786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:30.679488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba780-deae-4660-9e05-dfb44b2639e9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.684718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.685561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.685975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.686320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.686981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.688926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.722475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a567e-03fb-4982-a614-c53a86cc1ff6 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.727246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.727881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.728168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.728437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] PATH_INFO: `/v3/projects/b10dca5c2ea7435690051684c1113800` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.729622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10dca5c2ea7435690051684c1113800)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.731570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:30.823363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:30.834858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1022/2020] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:30 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:30.836394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:30.879220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10dca5c2ea7435690051684c1113800', 'request_id': 'req-f31e0120-e784-4464-bc2c-3c801127c454'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:30.888835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10dca5c2ea7435690051684c1113800', 'request_id': 'req-f31e0120-e784-4464-bc2c-3c801127c454'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:30.914822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac1a20-7e72-43a3-82a2-8e025e568b1c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:30.988929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:30.989666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:30.990255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:30.990466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:30.991248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:30.993414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.005909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10dca5c2ea7435690051684c1113800', 'request_id': 'req-f31e0120-e784-4464-bc2c-3c801127c454'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.027401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] The token cache is being invalidate because project b10dca5c2ea7435690051684c1113800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:31.027994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.028791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.030833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 999/2021] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:30 2022] DELETE /identity/v3/projects/b10dca5c2ea7435690051684c1113800 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:31.035556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.037847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:31.052342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e0120-e784-4464-bc2c-3c801127c454 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:31.057000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:31.059543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.059543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.059543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] PATH_INFO: `/v3/users/00c27ad41e774886a8e7626ce481b18d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.059543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] RBAC: Authorizing `identity:delete_user(user_id=00c27ad41e774886a8e7626ce481b18d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:31.061346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.161245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:31.169427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1023/2022] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:30 2022] POST /identity/v3/users => generated 300 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:31.179303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.183323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:31.208047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c6a47-d0f5-4023-b931-467113d05b70 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:31.213621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:31.214400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.214772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.215144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.215890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:31.219219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.237759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1024/2023] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:31.246111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.266709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8ecb-f1d8-41f2-ae39-f2995e967ae3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:31.277477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:31.278198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.278678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.279016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] PATH_INFO: `/v3/projects/c56ad0eeaefd41259ce67656e542be61/users/f67aee8ef0ea4801a44663b148a513ca/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.318242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56ad0eeaefd41259ce67656e542be61, user_id=f67aee8ef0ea4801a44663b148a513ca, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:31.325303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00c27ad41e774886a8e7626ce481b18d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.327365 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00c27ad41e774886a8e7626ce481b18d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.328681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.438910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00c27ad41e774886a8e7626ce481b18d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.448104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00c27ad41e774886a8e7626ce481b18d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.459535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1025/2024] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:31 2022] PUT /identity/v3/projects/c56ad0eeaefd41259ce67656e542be61/users/f67aee8ef0ea4801a44663b148a513ca/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:31.461852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.464962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:31.472845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00c27ad41e774886a8e7626ce481b18d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:31.482456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1000/2025] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:31 2022] DELETE /identity/v3/users/00c27ad41e774886a8e7626ce481b18d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:31.579052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d426ee-9129-4898-af85-9a8c9ad99479 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:31.657923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:31.658676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.659001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.659278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.660069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:31.661857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3cb19-4f56-47ae-b390-61e238ec8c35 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.679005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.682095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1026/2026] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:31.693423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9c1b8-9bb0-429f-9847-5da993fc87df admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:31.698515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:31.699207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.699506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.699820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] PATH_INFO: `/v3/projects/c56ad0eeaefd41259ce67656e542be61/users/f67aee8ef0ea4801a44663b148a513ca/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.701900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56ad0eeaefd41259ce67656e542be61, user_id=f67aee8ef0ea4801a44663b148a513ca, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:31.704727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:31.716789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1001/2027] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:31 2022] PUT /identity/v3/projects/c56ad0eeaefd41259ce67656e542be61/users/f67aee8ef0ea4801a44663b148a513ca/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:31.728700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:31.729026 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:31.729274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:31.762065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] MFA Rules not processed for user `f67aee8ef0ea4801a44663b148a513ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:31.763344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:31.883531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1027/2028] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:31.900676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:31.902657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:32.006601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb7fb2-1800-4b83-8446-9f9fca2e5032 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:32.010457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:32.011538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.015246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.015246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.024313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:32.025823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca3aa9-7284-411c-8dd9-a49452e67fa9 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:32.037005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1002/2029] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:32.072863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:32.105318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:32.145148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af078-78a3-466f-99cf-9737cdebb850 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:32.150880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:32.151525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.151826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.152093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] PATH_INFO: `/v3/users/95ac9bae7b534ec28f4d0341296d0f11` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.153461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ac9bae7b534ec28f4d0341296d0f11)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:32.157230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:32.276185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:32.298155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5084b-6311-45ff-94ba-c231f9e2df2a None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.298439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5084b-6311-45ff-94ba-c231f9e2df2a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.298730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5084b-6311-45ff-94ba-c231f9e2df2a None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.301216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1003/2030] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:26:32.333553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.334005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.334380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.365903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:32.369262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:32.441760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1004/2031] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:32.442955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:32.444352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:32.460654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ac9bae7b534ec28f4d0341296d0f11'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.461413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ac9bae7b534ec28f4d0341296d0f11'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.463188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6b574-8090-4a1a-b654-893baa9f156e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:32.468340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:32.468989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.469467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.469768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] PATH_INFO: `/v3/users/717423b700ad475792be81c05335560f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.470705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ac9bae7b534ec28f4d0341296d0f11'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.471505 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=717423b700ad475792be81c05335560f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:32.474018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:32.602430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ac9bae7b534ec28f4d0341296d0f11'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.604307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-dc2882ad-3a58-45b8-b64b-4da65188c16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ac9bae7b534ec28f4d0341296d0f11'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.631047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1028/2032] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:32 2022] DELETE /identity/v3/users/95ac9bae7b534ec28f4d0341296d0f11 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:32.635344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6865c-98fa-422e-8dc0-5c98f2599a25 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.635344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6865c-98fa-422e-8dc0-5c98f2599a25 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.635344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6865c-98fa-422e-8dc0-5c98f2599a25 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.635344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1029/2033] 10.222.0.99 () {54 vars in 961 bytes} [Fri Sep 2 21:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:26:32.636398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.636806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.637152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.638889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1030/2034] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:26:32.639586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:32.645652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:32.734015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51654ef9-0895-4dcb-b343-455c07fa8b62 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:32.740069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:32.740735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:32.741015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:32.741286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] PATH_INFO: `/v3/users/0b8cfb5d2a34451bb21d47bd5aae44f4` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:32.742514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8cfb5d2a34451bb21d47bd5aae44f4)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:32.744483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:32.795751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717423b700ad475792be81c05335560f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.796463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717423b700ad475792be81c05335560f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.925977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717423b700ad475792be81c05335560f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.933618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717423b700ad475792be81c05335560f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.945651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717423b700ad475792be81c05335560f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:32.951431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1005/2035] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:32 2022] DELETE /identity/v3/users/717423b700ad475792be81c05335560f => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:32.953572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:32.959540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.030094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920a3f1-1ca8-4941-b573-67dfa87c0fb6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.046639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.046639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.046639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.046639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] PATH_INFO: `/v3/projects/67bedceffddb46a4887f5b2b53cd0f02` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.047954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bedceffddb46a4887f5b2b53cd0f02)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.047954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.054984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8cfb5d2a34451bb21d47bd5aae44f4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.055826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8cfb5d2a34451bb21d47bd5aae44f4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.063474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8cfb5d2a34451bb21d47bd5aae44f4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.177557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8cfb5d2a34451bb21d47bd5aae44f4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.178241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec58c42c-3e45-469a-9f40-2277c19ec34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8cfb5d2a34451bb21d47bd5aae44f4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.197582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bedceffddb46a4887f5b2b53cd0f02', 'request_id': 'req-8622d221-ae97-44ef-983d-62a3796b18ad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.202568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1031/2036] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:32 2022] DELETE /identity/v3/users/0b8cfb5d2a34451bb21d47bd5aae44f4 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:33.202568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9a76-6829-4f9b-9a08-afad78a90ed0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.202568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9a76-6829-4f9b-9a08-afad78a90ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.202568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b9a76-6829-4f9b-9a08-afad78a90ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.207814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bedceffddb46a4887f5b2b53cd0f02', 'request_id': 'req-8622d221-ae97-44ef-983d-62a3796b18ad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.222273 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7b3b9a76-6829-4f9b-9a08-afad78a90ed0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:33.223525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7b3b9a76-6829-4f9b-9a08-afad78a90ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.287843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1032/2037] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:33.289638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e8e8-eee6-4486-9b68-52692252f378 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.289638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e8e8-eee6-4486-9b68-52692252f378 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.290019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e8e8-eee6-4486-9b68-52692252f378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.313758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e754e8e8-eee6-4486-9b68-52692252f378 None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:33.315560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e754e8e8-eee6-4486-9b68-52692252f378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.345412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bedceffddb46a4887f5b2b53cd0f02', 'request_id': 'req-8622d221-ae97-44ef-983d-62a3796b18ad'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.379437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] The token cache is being invalidate because project 67bedceffddb46a4887f5b2b53cd0f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:33.379751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.380514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8622d221-ae97-44ef-983d-62a3796b18ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.383854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1006/2038] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:32 2022] DELETE /identity/v3/projects/67bedceffddb46a4887f5b2b53cd0f02 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:33.385574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.386147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.386433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.392733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:33.403124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1033/2039] 10.222.0.99 () {58 vars in 1058 bytes} [Fri Sep 2 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:33.404692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.405131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.405382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.410195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] MFA Rules not processed for user `9af4ca265da3485cb7cec625913274ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:33.411437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.425847 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:33.433372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1007/2040] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:33.433372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.434090 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.434174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.435783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1034/2041] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:33.436674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.438027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.452104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb4873-bb95-478b-9c30-db788bc7d29a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.456998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.457631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.457931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.458199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] PATH_INFO: `/v3/users/44d369c414904bb9919749126e6ed197` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.459315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d369c414904bb9919749126e6ed197)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.461211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.506948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdae85-2f08-49cf-b36c-0f8e9f4d3a5d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.510601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.511470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.511755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.512031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.513846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.527490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.528959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.541577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1008/2042] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:33.542833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.559496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bdea4-59b9-46fc-878b-0c9cce02279c service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.566541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.570820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.570820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.570820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.570820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.582571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.584206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.594555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1009/2043] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:33.595403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.597428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.610959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6155f0-c93b-4729-ae3f-8733f0355180 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.621619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.622666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.622666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.623265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.644142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.644800 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.656433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1010/2044] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:33.657300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.668590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b438b-44ac-4421-98c8-c16a3fba6518 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.678673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.679989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.679989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.680175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.692185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.693640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.707999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1011/2045] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:33.708857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:33.710214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:33.722816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613754c-e4a0-46d0-b0b8-4f74089e30c0 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:33.727964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:33.730439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:33.730439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:33.730439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] PATH_INFO: `/v3/users/46c6a073213343c89c55e965e9828684` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:33.730439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c6a073213343c89c55e965e9828684)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:33.732159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:33.935344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d369c414904bb9919749126e6ed197'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.936128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d369c414904bb9919749126e6ed197'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.943783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d369c414904bb9919749126e6ed197'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:33.948705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:34.062008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d369c414904bb9919749126e6ed197'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.063194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d369c414904bb9919749126e6ed197'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.086819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1035/2046] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:33 2022] DELETE /identity/v3/users/44d369c414904bb9919749126e6ed197 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:34.091292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:34.093024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.095321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c6a073213343c89c55e965e9828684'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.096530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c6a073213343c89c55e965e9828684'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.190532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f320d22-4a99-43a4-a0fa-f41995d1fb65 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:34.196813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:34.197477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:34.197786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:34.198072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] PATH_INFO: `/v3/projects/4b972cee2d99491096a0aba46391cec6` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:34.199313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b972cee2d99491096a0aba46391cec6)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:34.201612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:34.324942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c6a073213343c89c55e965e9828684'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.332978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c6a073213343c89c55e965e9828684'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.345765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c6a073213343c89c55e965e9828684'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.358583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1012/2047] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:33 2022] DELETE /identity/v3/users/46c6a073213343c89c55e965e9828684 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:34.359467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:34.364896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.393450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b972cee2d99491096a0aba46391cec6', 'request_id': 'req-aa40e845-78a9-4f1f-aaef-98c7a20a0529'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.409548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b972cee2d99491096a0aba46391cec6', 'request_id': 'req-aa40e845-78a9-4f1f-aaef-98c7a20a0529'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.453424 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034043ff-07a8-4463-aed6-1579fc8c8d30 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:34.518925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b972cee2d99491096a0aba46391cec6', 'request_id': 'req-aa40e845-78a9-4f1f-aaef-98c7a20a0529'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.523942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:34.524888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:34.525498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:34.525827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] PATH_INFO: `/v3/projects/d06efda7d2894ac684959b8fb7e8f0bd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:34.527026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06efda7d2894ac684959b8fb7e8f0bd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:34.530179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:34.535242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] The token cache is being invalidate because project 4b972cee2d99491096a0aba46391cec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:34.535930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.538047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.540512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1036/2048] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:34 2022] DELETE /identity/v3/projects/4b972cee2d99491096a0aba46391cec6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:34.541956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:34.548187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.564181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40e845-78a9-4f1f-aaef-98c7a20a0529 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:34.569994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:34.570655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:34.570930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:34.571200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] PATH_INFO: `/v3/projects/d45a494827004cb5a392662d8a6a727a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:34.572355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45a494827004cb5a392662d8a6a727a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:34.574358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:34.755948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06efda7d2894ac684959b8fb7e8f0bd', 'request_id': 'req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.762401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06efda7d2894ac684959b8fb7e8f0bd', 'request_id': 'req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.801276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45a494827004cb5a392662d8a6a727a', 'request_id': 'req-adcb5766-445b-4016-9722-807465864d40'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.809034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45a494827004cb5a392662d8a6a727a', 'request_id': 'req-adcb5766-445b-4016-9722-807465864d40'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.855826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06efda7d2894ac684959b8fb7e8f0bd', 'request_id': 'req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.866880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] The token cache is being invalidate because project d06efda7d2894ac684959b8fb7e8f0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:34.867095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.867726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5fe5c9b6-d9a6-4344-9cf2-0310088d125c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.869220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1013/2049] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:34 2022] DELETE /identity/v3/projects/d06efda7d2894ac684959b8fb7e8f0bd => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:34.870441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:34.870718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:34.870983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:34.891872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:34.893707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.956171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1014/2050] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:34.957073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:34.958408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.964596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45a494827004cb5a392662d8a6a727a', 'request_id': 'req-adcb5766-445b-4016-9722-807465864d40'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.973529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8306f-0e6e-479a-8bdb-d31b3c819127 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:34.977828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] The token cache is being invalidate because project d45a494827004cb5a392662d8a6a727a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:34.978141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.978788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:34.978936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:34.979610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:34.980058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:34.980203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] PATH_INFO: `/v3/projects/a96eea4543f94dd18842d831b931da33` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:34.980890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1037/2051] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:34 2022] DELETE /identity/v3/projects/d45a494827004cb5a392662d8a6a727a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:34.982187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:34.983587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96eea4543f94dd18842d831b931da33)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:34.983719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:34.985461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:34.997031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb5766-445b-4016-9722-807465864d40 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:35.001972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:35.003239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.003239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.003239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] PATH_INFO: `/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.004747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e711a2b5b64e7a8384e50c83e944ac)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:35.006754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:35.086376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:35.159784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96eea4543f94dd18842d831b931da33', 'request_id': 'req-3cc167f2-3361-407a-9cfb-702a7e7c61a0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.166434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96eea4543f94dd18842d831b931da33', 'request_id': 'req-3cc167f2-3361-407a-9cfb-702a7e7c61a0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.207152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e711a2b5b64e7a8384e50c83e944ac', 'request_id': 'req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.214368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e711a2b5b64e7a8384e50c83e944ac', 'request_id': 'req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.255984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96eea4543f94dd18842d831b931da33', 'request_id': 'req-3cc167f2-3361-407a-9cfb-702a7e7c61a0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.268965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] The token cache is being invalidate because project a96eea4543f94dd18842d831b931da33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:35.269242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.269951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.271640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1015/2052] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:34 2022] DELETE /identity/v3/projects/a96eea4543f94dd18842d831b931da33 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:35.274602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:35.274602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:35.346713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc167f2-3361-407a-9cfb-702a7e7c61a0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:35.366153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e711a2b5b64e7a8384e50c83e944ac', 'request_id': 'req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.377211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] The token cache is being invalidate because project f2e711a2b5b64e7a8384e50c83e944ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:35.377978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.378683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5752dda4-caf2-4f18-a3eb-3d6c6104e47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.380206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1038/2053] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:34 2022] DELETE /identity/v3/projects/f2e711a2b5b64e7a8384e50c83e944ac => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:35.381472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.381881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.382089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.402646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:35.403851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:35.408301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:35.409744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.410065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.410377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] PATH_INFO: `/v3/users/f67aee8ef0ea4801a44663b148a513ca` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.411545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67aee8ef0ea4801a44663b148a513ca)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:35.412336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1039/2054] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:35.413222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:35.413476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:35.414572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:35.426691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f8499-1f66-48cb-b7a6-78ed4fe91420 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:35.431458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:35.432082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.432359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.432628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] PATH_INFO: `/v3/users/5ad5019fc71f4fdd92e426ecb14ebd9d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.433784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad5019fc71f4fdd92e426ecb14ebd9d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:35.435724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:35.672159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad5019fc71f4fdd92e426ecb14ebd9d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.672422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67aee8ef0ea4801a44663b148a513ca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.672930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad5019fc71f4fdd92e426ecb14ebd9d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.674132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67aee8ef0ea4801a44663b148a513ca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.681922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad5019fc71f4fdd92e426ecb14ebd9d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.782764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67aee8ef0ea4801a44663b148a513ca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.790276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67aee8ef0ea4801a44663b148a513ca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.802451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67aee8ef0ea4801a44663b148a513ca'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.808521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1016/2055] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:35 2022] DELETE /identity/v3/users/f67aee8ef0ea4801a44663b148a513ca => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:35.809302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:35.810667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:35.874357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad5019fc71f4fdd92e426ecb14ebd9d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.874993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c7c73c82-1ce4-48c2-8346-3f01352bbb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad5019fc71f4fdd92e426ecb14ebd9d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:35.879142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a86fd1-202d-478c-bf51-74649076bec0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:35.884670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:35.885568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.885886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.886428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] PATH_INFO: `/v3/users/950cd0f1ec58470692911732a747a227` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.887746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=950cd0f1ec58470692911732a747a227)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:35.889738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:35.896957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1040/2056] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:35 2022] DELETE /identity/v3/users/5ad5019fc71f4fdd92e426ecb14ebd9d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:35.901394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:35.901682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:35.902104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:35.932202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:35.933357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.000019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1041/2057] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:36.001274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.003218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.015390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be409e-7770-4393-9607-d48cd59d5a60 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.020415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.021025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.021300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.021615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.022301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:36.022603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.024219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3f41a-41ed-4c6a-bca7-1993baebda1b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.042107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1042/2058] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:36.043899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.043899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.044139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.071282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:36.071805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.080818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1043/2059] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:36.081843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.093515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67005f-f83f-4481-a8ce-33006dbc7760 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.101864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.102812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.103303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.103754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.104819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.107024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.153462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:36.228749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950cd0f1ec58470692911732a747a227'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.229536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950cd0f1ec58470692911732a747a227'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.269809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1044/2060] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:36 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:36.270927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.273497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.342982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950cd0f1ec58470692911732a747a227'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.348776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f5e70-a7e5-4658-b0ea-715845593c3f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.351731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950cd0f1ec58470692911732a747a227'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.353479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.354115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.354481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.354676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.355309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:36.355601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.357185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.364866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950cd0f1ec58470692911732a747a227'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.369146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1045/2061] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:36.369994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.371252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1017/2062] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:35 2022] DELETE /identity/v3/users/950cd0f1ec58470692911732a747a227 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:36.373853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.374020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.388199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5805-4eab-46f2-85fb-84cfbbf97f08 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.443133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311e851-655b-4ee5-a8b2-7936f7018a1c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.447931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.448558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.455291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.455291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.455291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.455291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.459959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.460641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.460927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.461197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.462003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.464387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.612577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1046/2063] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:36 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:36.614480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.616130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1018/2064] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:36 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:36.618232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:36.621178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.621347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:36.638735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39115b39-1634-44b1-8d77-b47ae045a870 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.707204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.707489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b949aec-f9d8-4679-be23-c46a8ff6b0eb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:36.708064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.708490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.708865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] PATH_INFO: `/v3/projects/3606045986774fb4b9cfe0fe587670d0` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.710428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3606045986774fb4b9cfe0fe587670d0)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.712973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.713199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:36.713673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:36.720384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:36.720384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] PATH_INFO: `/v3/projects/c56ad0eeaefd41259ce67656e542be61` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:36.721796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56ad0eeaefd41259ce67656e542be61)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:36.723794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:36.956655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3606045986774fb4b9cfe0fe587670d0', 'request_id': 'req-d9eccd63-0844-4457-bad0-612717417b46'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.957369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56ad0eeaefd41259ce67656e542be61', 'request_id': 'req-7a4f160e-b43a-4344-9400-946e679482b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.963374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3606045986774fb4b9cfe0fe587670d0', 'request_id': 'req-d9eccd63-0844-4457-bad0-612717417b46'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:36.963615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56ad0eeaefd41259ce67656e542be61', 'request_id': 'req-7a4f160e-b43a-4344-9400-946e679482b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.044776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:37.079431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3606045986774fb4b9cfe0fe587670d0', 'request_id': 'req-d9eccd63-0844-4457-bad0-612717417b46'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.091484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] The token cache is being invalidate because project 3606045986774fb4b9cfe0fe587670d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:37.091800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.092448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.094012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1047/2065] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:36 2022] DELETE /identity/v3/projects/3606045986774fb4b9cfe0fe587670d0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:37.094883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.096330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.141385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56ad0eeaefd41259ce67656e542be61', 'request_id': 'req-7a4f160e-b43a-4344-9400-946e679482b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.153277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] The token cache is being invalidate because project c56ad0eeaefd41259ce67656e542be61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:37.153277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.153979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.155507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1019/2066] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:36 2022] DELETE /identity/v3/projects/c56ad0eeaefd41259ce67656e542be61 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:37.156256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.157737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.168695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eccd63-0844-4457-bad0-612717417b46 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.227988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f160e-b43a-4344-9400-946e679482b0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.228538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.229589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.229928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.230200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.230848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.232679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.232944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.233608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.233920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.234190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.234823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.236731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab283920-b7da-47d9-9187-0121616c9071 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.244774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1048/2067] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:37.244985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.246597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.259833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b3b12-a52b-454f-a9b6-940174c4156f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.264835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.265482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.265766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.266028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] PATH_INFO: `/v3/projects/9925a33340a24bc3b9988f3e6d2879e9` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.267131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9925a33340a24bc3b9988f3e6d2879e9)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.269309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.373968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1020/2068] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:37.375939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519621-1d06-4a7e-b3b0-f926e9d8d8e1 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.376297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519621-1d06-4a7e-b3b0-f926e9d8d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.376497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519621-1d06-4a7e-b3b0-f926e9d8d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.392815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9925a33340a24bc3b9988f3e6d2879e9', 'request_id': 'req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.399773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-36519621-1d06-4a7e-b3b0-f926e9d8d8e1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.403864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36519621-1d06-4a7e-b3b0-f926e9d8d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.404018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9925a33340a24bc3b9988f3e6d2879e9', 'request_id': 'req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.470435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1021/2069] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.472417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.472704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.472971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.497242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.498411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.506907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1022/2070] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.507623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.508917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.522937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491b2c-f28f-46fe-aa90-667acf665313 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.528745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.529395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.529914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.530187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.530839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.532463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.542118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9925a33340a24bc3b9988f3e6d2879e9', 'request_id': 'req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.552754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1023/2071] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:37.552754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.557476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] The token cache is being invalidate because project 9925a33340a24bc3b9988f3e6d2879e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:37.557857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.558579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-86d1c010-96a2-46fa-a2fa-d9907d7ce43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:37.559444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb0678-a39a-472b-bc7a-fcd225e1870c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.560949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1049/2072] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:37 2022] DELETE /identity/v3/projects/9925a33340a24bc3b9988f3e6d2879e9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:37.561552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.561959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.562292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.566861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.567702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.568165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.568427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.569113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.571923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ea480-a27a-4b7a-9964-2dae4c4d780a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.584668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.584932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1024/2073] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:37.586051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.591200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9923-c67b-4146-92d1-8eaac3e03637 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.591724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9923-c67b-4146-92d1-8eaac3e03637 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.592173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d9923-c67b-4146-92d1-8eaac3e03637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.596841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1050/2074] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.596841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.600739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.610361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd056223-e9ef-49f3-a225-42121afa8f17 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.618598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ab9d9923-c67b-4146-92d1-8eaac3e03637 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.620284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ab9d9923-c67b-4146-92d1-8eaac3e03637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.623457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.624278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.624732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.625212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] PATH_INFO: `/v3/projects/de8efd74da71400a83f05127451c13fd/users/7d2ee46e8302468bb90ee913eeeba4af/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.632564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1025/2075] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.640938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.640938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.640938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.644286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8efd74da71400a83f05127451c13fd, user_id=7d2ee46e8302468bb90ee913eeeba4af, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.647892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678056a-97d3-4ea5-aa50-c951c43da926 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.660037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.661218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.670226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1026/2076] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.673213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.673213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.685571 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d306c2-5a24-4ced-9c2e-fdd8e76b6879 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.691220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.691220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.691220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.691629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.692136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.693824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.703959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1027/2077] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:37.707601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.714799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d663d16-de76-4ed3-b998-2f9f48898ef4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.719473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.720043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.720286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.720556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.721707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:37.722105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.723717 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.735842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1028/2078] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:37.736650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.746863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6da848-8437-4a89-a3c1-c1e0774f8883 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.751490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.752001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.752280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.752594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.753361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:37.753663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.755263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.759805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1051/2079] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:37 2022] PUT /identity/v3/projects/de8efd74da71400a83f05127451c13fd/users/7d2ee46e8302468bb90ee913eeeba4af/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:37.763001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.763001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.763001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.767587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1029/2080] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:37.768492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.778789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b91283-6040-409d-bce7-376d383e807d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.783183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.785493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.835936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:37.839371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.840036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.840316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.840590 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] PATH_INFO: `/v3/projects/a12ed1b1ebff4fb3a1522cc79fe54e49/users/7b8c60a410394867a333f0febb06cda8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.845279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1052/2081] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:37.846150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:37.856172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9836af-88d2-4da9-aca2-444934de67c2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:37.856379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12ed1b1ebff4fb3a1522cc79fe54e49, user_id=7b8c60a410394867a333f0febb06cda8, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.859093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e57dd-9e28-427c-9b66-ddb50021bea9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.860935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:37.861531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.861853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.862117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.863023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:37.864868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:37.937539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1030/2082] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:37 2022] PUT /identity/v3/projects/a12ed1b1ebff4fb3a1522cc79fe54e49/users/7b8c60a410394867a333f0febb06cda8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:37.946836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:37.946836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:37.946836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:37.959783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:37.960988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:37.983546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1053/2083] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:37.985891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.035095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1031/2084] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:38.035095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.043193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a097c-9fae-4dbc-9058-f8be63f93a66 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.048306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.048940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.049053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64902e-724d-457f-b5ba-246678e6d164 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.049210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.049525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.051787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:38.051787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.052332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa695470-7e8a-4a35-bf9f-442f5e3e85f5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.057639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.058734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.059236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.059797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.061036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.064628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1032/2085] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:38.064907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.066030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.078456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.078456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.098844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] MFA Rules not processed for user `7b8c60a410394867a333f0febb06cda8`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:38.100115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:38.161325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1054/2086] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:37 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:38.162432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.166245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1033/2087] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:38.167029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.168427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:38.228341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a801509-a6f4-497c-8984-eef0bf4446ca None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.233908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c481bd-0bd7-4391-bb39-62103cd21e88 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.238860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.239405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.239706 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.239951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.240585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.242597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.275063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:38.280086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.280808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.282532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.282532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.290578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.291951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.301301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1034/2088] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:38.303177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.311911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e4fc8-b56d-4ddf-9dac-a1c82f34a2e5 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.317772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.318385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.318665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.318936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.319572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.321479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.341877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1055/2089] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:38.342840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.352678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851590c-b2e0-463a-82fa-3817c3aa592e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.357382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.357997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.358282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.358547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.359161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.361066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.470211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1056/2090] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/users => generated 281 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:38.475629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.475754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1035/2091] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:38.476029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:38.477075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.551146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1de2-519d-4967-9da2-096498df8c10 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.551378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfe216-127c-4fbd-ad9b-4dbefea40976 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.551378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.551378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.551378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.551748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.551748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.552700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.562752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1036/2092] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:38.563728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.573468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c04e-9f0f-4802-aa55-70b716b1ed47 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.578187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.578777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.579052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.579328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.579957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.582251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.592722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1037/2093] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:38.593797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.603149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.604113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.604657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ef5e4-3a3e-4c6c-8e60-aec5eb8b2f45 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.604880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.605298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.621266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.622748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.626221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.626221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.626221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.626221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.679459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.680973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.691132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1057/2094] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:38.692002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.705424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082173-2751-4c31-87be-d55926c8609a service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.712698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.713317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.713641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.713948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.714864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.717306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.729478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1058/2095] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:38.730553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.740929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aed5c7-b2a4-429d-b8a6-da600e073677 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.748879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.750432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.750718 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.751028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.751678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.752350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1038/2096] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:38.753368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.753664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.766871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944497-6db2-4919-ba8e-1e0cc47174e0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.773161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1059/2097] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:38.774553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.781979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.781979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.781979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.781979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] PATH_INFO: `/v3/projects/7fb093ab109642dc883861a82285e709/users/c94bfb59c02046378af4e1bc6650c15e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.791149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750763-1030-4039-959e-5d5d1cb5754c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.797178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb093ab109642dc883861a82285e709, user_id=c94bfb59c02046378af4e1bc6650c15e, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.798390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.799094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.799541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.799885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.800661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.801051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.808650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.822401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1060/2098] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:38.822401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.833498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aca6e0-3fec-4979-aaf0-9406b15614b4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:38.839779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:38.840460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:38.840733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:38.841005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] PATH_INFO: `/v3/projects/36e8b280d1964d048e3549b6ab1c1393/users/52a1a3a069c743d4b431e53d20a93b3a/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:38.860368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e8b280d1964d048e3549b6ab1c1393, user_id=52a1a3a069c743d4b431e53d20a93b3a, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:38.863072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d9e15-9fe7-49e8-9a6d-6309f71068b9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:38.970781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1039/2099] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:38 2022] PUT /identity/v3/projects/7fb093ab109642dc883861a82285e709/users/c94bfb59c02046378af4e1bc6650c15e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:38.979104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:38.997289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:39.000067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1061/2100] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:38 2022] PUT /identity/v3/projects/36e8b280d1964d048e3549b6ab1c1393/users/52a1a3a069c743d4b431e53d20a93b3a/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:39.000213 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63add1a-e0a3-4d23-9f49-8035c4e8fa36 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.000434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63add1a-e0a3-4d23-9f49-8035c4e8fa36 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.000701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63add1a-e0a3-4d23-9f49-8035c4e8fa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.042349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f63add1a-e0a3-4d23-9f49-8035c4e8fa36 None None] MFA Rules not processed for user `c94bfb59c02046378af4e1bc6650c15e`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:39.043788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f63add1a-e0a3-4d23-9f49-8035c4e8fa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:39.068211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d32f1-acca-4c75-8687-63024b9964c4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:39.128550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1062/2101] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:39.128550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.128550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.128550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.145982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:39.146595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.146879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.147155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.147810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:39.149447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:39.159153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] MFA Rules not processed for user `52a1a3a069c743d4b431e53d20a93b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:39.160170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1040/2102] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:39.160834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:39.162638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:39.163750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:39.226530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540baea-36c4-4476-9e79-dd82e4893f50 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:39.232119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:39.232740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.233038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.233493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.247401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:39.249292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fff0-7bc6-4225-906e-821ebb2e6184 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:39.256298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1063/2103] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:39.256935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:39.262001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1041/2104] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:39.275746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe3ccc-fbb2-4190-93ef-699f3e75d0b2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:39.280308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:39.281059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.281344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.281613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] PATH_INFO: `/v3/projects/228e69f3738f4c61a27dab62d5c3781d/users/051d70cc75f140e6b7e1c855266014d8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.307094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=228e69f3738f4c61a27dab62d5c3781d, user_id=051d70cc75f140e6b7e1c855266014d8, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:39.310188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:39.392638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:39.399866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1064/2105] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:39 2022] PUT /identity/v3/projects/228e69f3738f4c61a27dab62d5c3781d/users/051d70cc75f140e6b7e1c855266014d8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:39.410751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c98c-a968-4462-87fc-722955601310 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.411314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c98c-a968-4462-87fc-722955601310 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.411525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477c98c-a968-4462-87fc-722955601310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.443943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4477c98c-a968-4462-87fc-722955601310 None None] MFA Rules not processed for user `051d70cc75f140e6b7e1c855266014d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:39.445332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4477c98c-a968-4462-87fc-722955601310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:39.527827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1042/2106] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:39.539546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:39.541420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:39.640489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc2070-4fe7-464c-ae8e-68bf1f582a01 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:39.645094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:39.645807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:39.646086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:39.646357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:39.655260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:39.656703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:39.665740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1065/2107] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:40.342951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477c98c-a968-4462-87fc-722955601310 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:40.410793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477c98c-a968-4462-87fc-722955601310 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:40.412490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:40.413349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:40.414169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:40.414448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:40.414719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:40.475822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:40.477400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5cc5d-9af0-4cbb-be00-22af14b83eac service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:40.486734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1043/2108] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:40.508014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:40.606173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ed44-bfc4-4447-b0ad-5ed27fdefe55 service glance] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:40.611782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:40.612797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:40.613076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:40.613371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] PATH_INFO: `/v3/users/52a1a3a069c743d4b431e53d20a93b3a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:40.614647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a1a3a069c743d4b431e53d20a93b3a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:40.616528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:40.957905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a1a3a069c743d4b431e53d20a93b3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:40.958654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a1a3a069c743d4b431e53d20a93b3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:40.965952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a1a3a069c743d4b431e53d20a93b3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:41.035772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a1a3a069c743d4b431e53d20a93b3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:41.037637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a1a3a069c743d4b431e53d20a93b3a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:41.076251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1066/2109] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:40 2022] DELETE /identity/v3/users/52a1a3a069c743d4b431e53d20a93b3a => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:41.080879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:41.082950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:41.083265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:41.117860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:41.119753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:41.189951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1044/2110] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:41.198669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:41.200112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:41.295605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff977a-1ba3-48fd-ab8b-40e8348dff3a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:41.299987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:41.300690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:41.301063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:41.301478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:41.311585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:41.313124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b58d11-8028-42ee-9511-fae5d9eae4cd service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:41.324427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1067/2111] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:42.101924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:42.104609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:42.131546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef434fb-94a2-47ee-b67e-28eb6b780e62 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:42.138240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:42.139174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:42.139601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:42.140067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] PATH_INFO: `/v3/projects/36e8b280d1964d048e3549b6ab1c1393` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:42.142004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e8b280d1964d048e3549b6ab1c1393)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:42.146585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:42.359803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e8b280d1964d048e3549b6ab1c1393', 'request_id': 'req-b70ff811-3e04-47c5-9689-4938a134f3ab'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:42.370398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e8b280d1964d048e3549b6ab1c1393', 'request_id': 'req-b70ff811-3e04-47c5-9689-4938a134f3ab'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:42.445781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:42.484076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e8b280d1964d048e3549b6ab1c1393', 'request_id': 'req-b70ff811-3e04-47c5-9689-4938a134f3ab'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:42.507375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] The token cache is being invalidate because project 36e8b280d1964d048e3549b6ab1c1393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:42.507979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:42.509484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b70ff811-3e04-47c5-9689-4938a134f3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:42.513083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1045/2112] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:42 2022] DELETE /identity/v3/projects/36e8b280d1964d048e3549b6ab1c1393 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:42.517994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:42.518515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:42.519024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:42.544082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:42.545654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:42.637389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1068/2113] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:42.645009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:42.645819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:42.646316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:42.681381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:42.683853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:42.698414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1046/2114] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:42.702608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:42.729764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290d06d-5bbb-4885-9a08-753278eac116 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:42.729764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:42.729764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:42.729764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:42.730108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:42.730108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:42.730108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:42.730108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:42.745801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1069/2115] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:42.749967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:42.765951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea9723-5f93-4c3c-b375-d426a5ff8746 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:42.771390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:42.772219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:42.772685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:42.773136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:42.773978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:42.776093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:42.991348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1047/2116] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:42 2022] POST /identity/v3/projects => generated 373 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:42.996363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:43.010029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b526-8b8f-4748-b47b-9d7724675b8c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:43.084681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:43.085311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.085585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.086060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.086702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:43.088638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:43.287238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1070/2117] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:42 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:43.291275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:43.320479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb8223-100c-4131-b54a-1b029193522d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:43.334305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:43.335193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.335649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.336117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.336934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:43.339241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:43.352821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1048/2118] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:43.356900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:43.378631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681856e-6545-4725-8500-bd2c6d13e857 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:43.385154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:43.391437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.391861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.392251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.393922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:43.396053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90243b-56c1-49c4-8ba8-6f7214ccbbba admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:43.413426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1071/2119] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:43.414587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:43.425486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b1936-2caf-4989-89b6-fd421ca4576e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:43.430512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:43.431172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.431676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.431959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] PATH_INFO: `/v3/projects/9a4c020ce4554d3496ce5c6a306a4c21/users/14e14f99e7194107a07bf6623967b942/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.455894 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4c020ce4554d3496ce5c6a306a4c21, user_id=14e14f99e7194107a07bf6623967b942, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:43.458858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:43.661640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1049/2120] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:43 2022] PUT /identity/v3/projects/9a4c020ce4554d3496ce5c6a306a4c21/users/14e14f99e7194107a07bf6623967b942/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:43.673011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b9ac-cf8e-4d52-b828-23639df90057 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.673568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b9ac-cf8e-4d52-b828-23639df90057 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.673801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398b9ac-cf8e-4d52-b828-23639df90057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.714550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8398b9ac-cf8e-4d52-b828-23639df90057 None None] MFA Rules not processed for user `14e14f99e7194107a07bf6623967b942`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:43.716167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8398b9ac-cf8e-4d52-b828-23639df90057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:43.799677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1072/2121] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:43.813736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:43.815582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:43.881982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed935df5-d314-482a-9d4b-fc5ed0784b58 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:43.886240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:43.886615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:43.887561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:43.888098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:43.902109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:43.902109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:43.911294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1050/2122] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:44.423954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:45.688490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:45.855149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404de7b8-12ad-4865-8fa1-178f10f85f67 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:45.855427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404de7b8-12ad-4865-8fa1-178f10f85f67 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:45.855696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404de7b8-12ad-4865-8fa1-178f10f85f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:45.884189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-404de7b8-12ad-4865-8fa1-178f10f85f67 None None] MFA Rules not processed for user `7d2ee46e8302468bb90ee913eeeba4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:45.885393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-404de7b8-12ad-4865-8fa1-178f10f85f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:45.959012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1073/2123] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:45.972993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:45.974486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:46.050348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563fcf0-0321-47e6-9230-cfd1b5816fa9 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:46.056110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:46.056721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:46.057001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:46.057311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:46.066521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:46.078188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:46.078871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1051/2124] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:47.921834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:49.007348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:49.590930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07404dd3-a3da-42a2-9ef5-37072f3aec6d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:49.592164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07404dd3-a3da-42a2-9ef5-37072f3aec6d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:49.592164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07404dd3-a3da-42a2-9ef5-37072f3aec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:49.614538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-07404dd3-a3da-42a2-9ef5-37072f3aec6d None None] MFA Rules not processed for user `7d2ee46e8302468bb90ee913eeeba4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:49.617247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-07404dd3-a3da-42a2-9ef5-37072f3aec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:49.623781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1074/2125] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:50.533001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:51.693626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:52.491613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:53.785485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:54.480194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:54.615543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bcb00-c28a-424a-8415-55c3bd6c5356 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:54.620402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:54.621147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:54.621496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:54.621876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] PATH_INFO: `/v3/users/14e14f99e7194107a07bf6623967b942` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:54.623139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e14f99e7194107a07bf6623967b942)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:54.626589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:54.683966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:54.934955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e14f99e7194107a07bf6623967b942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:54.935937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e14f99e7194107a07bf6623967b942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:55.160639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e14f99e7194107a07bf6623967b942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:55.168049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e14f99e7194107a07bf6623967b942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:55.181199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e14f99e7194107a07bf6623967b942'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:55.200643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1052/2126] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:54 2022] DELETE /identity/v3/users/14e14f99e7194107a07bf6623967b942 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:55.202414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8abd9-6038-4359-a0a0-6c9b0ad5f371 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:55.202414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8abd9-6038-4359-a0a0-6c9b0ad5f371 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:55.202414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8abd9-6038-4359-a0a0-6c9b0ad5f371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:55.216342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-07f8abd9-6038-4359-a0a0-6c9b0ad5f371 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:55.217636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-07f8abd9-6038-4359-a0a0-6c9b0ad5f371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:55.304408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1075/2127] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:55.314367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:55.322979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:55.380159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af31ca-8858-455d-934c-dcd10000f23d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:55.384029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:55.384670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:55.385063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:55.385341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:55.403975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:55.403975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:55.404454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac292d-5ff3-466b-8eb0-c15a08241c3b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:55.404454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac292d-5ff3-466b-8eb0-c15a08241c3b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:55.404454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac292d-5ff3-466b-8eb0-c15a08241c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:55.411258 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1053/2128] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:55.424753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8bac292d-5ff3-466b-8eb0-c15a08241c3b None None] MFA Rules not processed for user `dc0bc75b6b3f4d1689bc1f7faaa2e5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:55.426135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8bac292d-5ff3-466b-8eb0-c15a08241c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:55.491691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1076/2129] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:55.617242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:55.620116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:55.637290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979d95a-2787-43e3-ab13-064094806a3a service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:55.643446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:55.645157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:55.645585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:55.646028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] PATH_INFO: `/v3/users/c94bfb59c02046378af4e1bc6650c15e` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:55.647863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94bfb59c02046378af4e1bc6650c15e)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:55.650346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:55.930334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94bfb59c02046378af4e1bc6650c15e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:55.931001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94bfb59c02046378af4e1bc6650c15e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.017593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94bfb59c02046378af4e1bc6650c15e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.026456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94bfb59c02046378af4e1bc6650c15e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.040806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94bfb59c02046378af4e1bc6650c15e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.048453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1054/2130] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:26:55 2022] DELETE /identity/v3/users/c94bfb59c02046378af4e1bc6650c15e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:56.056914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.057531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.060115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.080842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:56.094690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:56.095929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:56.178286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:56.180487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:56.188682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1077/2131] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:56.205738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87d924-c277-43d7-b640-b28277fa9939 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:56.213565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:56.214591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.216031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.216311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] PATH_INFO: `/v3/projects/9a4c020ce4554d3496ce5c6a306a4c21` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.218041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4c020ce4554d3496ce5c6a306a4c21)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:56.220249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:56.233654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:56.237860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:56.341856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3065be-5eff-42b3-b30f-be8835972767 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:56.346110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:56.346712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.347321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.349022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.361541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:56.364476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66470-92c3-437c-987b-49c7c0ea7d63 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:56.387663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1078/2132] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:26:56.437496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4c020ce4554d3496ce5c6a306a4c21', 'request_id': 'req-725da768-65df-4942-b4c6-b62090f5d9a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.446120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4c020ce4554d3496ce5c6a306a4c21', 'request_id': 'req-725da768-65df-4942-b4c6-b62090f5d9a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.594817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4c020ce4554d3496ce5c6a306a4c21', 'request_id': 'req-725da768-65df-4942-b4c6-b62090f5d9a4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.608223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] The token cache is being invalidate because project 9a4c020ce4554d3496ce5c6a306a4c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:56.608515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.609461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-725da768-65df-4942-b4c6-b62090f5d9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:56.613997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1055/2133] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:56 2022] DELETE /identity/v3/projects/9a4c020ce4554d3496ce5c6a306a4c21 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:26:56.620983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.621638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.623229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.662862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:56.664618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:56.795832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:56.829388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1079/2134] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:56.848909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.849532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.850122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.882613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:56.883923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:56.895774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1056/2135] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:56.901745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:56.926540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a687da-9917-48a5-aa09-0642700c96cd None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:56.934671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:56.935325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.935822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.938150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.939504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:56.939905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:56.942289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:56.956136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1080/2136] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:26:56.961242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:56.971309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974db94-0779-4acd-b692-307e80e60b21 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:56.976322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:56.976932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:56.977204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:56.977467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:56.978154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:56.980098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.060174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.062073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.081928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf8b9f-023c-4644-809f-2607bece6eb3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.095027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.095941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.096116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.096386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] PATH_INFO: `/v3/projects/7fb093ab109642dc883861a82285e709` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.098135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb093ab109642dc883861a82285e709)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.100051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.143630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1057/2137] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:56 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:57.145578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.216105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0728c-6cb6-4845-b363-d399147cc25d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.221413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.222088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.222372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.227782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.227782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.227782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.307555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb093ab109642dc883861a82285e709', 'request_id': 'req-c1ebd448-03d7-471d-885d-57836dd005bf'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:57.322577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1058/2138] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:57.323148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb093ab109642dc883861a82285e709', 'request_id': 'req-c1ebd448-03d7-471d-885d-57836dd005bf'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:57.326824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.340395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2997d-d6bb-4e95-bc3e-371f238a4525 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.421007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.421758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.422112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.424059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.424059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.427617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.434136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb093ab109642dc883861a82285e709', 'request_id': 'req-c1ebd448-03d7-471d-885d-57836dd005bf'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:57.445878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1059/2139] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:26:57.451505 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.454778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] The token cache is being invalidate because project 7fb093ab109642dc883861a82285e709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:26:57.455166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:57.456129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c1ebd448-03d7-471d-885d-57836dd005bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:26:57.458773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1081/2140] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:26:57 2022] DELETE /identity/v3/projects/7fb093ab109642dc883861a82285e709 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:57.465445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d361c-0415-4489-8564-e49f30b9c553 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.466051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d361c-0415-4489-8564-e49f30b9c553 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.466479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d361c-0415-4489-8564-e49f30b9c553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.469584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe366ec0-996a-495d-89e7-8674c9bbee98 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.476282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.476862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.477153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.477522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.478217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.479955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.495679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-604d361c-0415-4489-8564-e49f30b9c553 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:57.497427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-604d361c-0415-4489-8564-e49f30b9c553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.502908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1060/2141] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:57.504968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.507183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.507988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1082/2142] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:57.517947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.518231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.519696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.522675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1f274-4ffd-44e3-852a-13680905e587 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.530554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.531250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.531531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.531803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] PATH_INFO: `/v3/projects/cbffdee4dc4341e4a33fcfe884f28072/users/50dab514c64d47d095bd6d1ac8ce3b56/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.547431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:57.548712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.552623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbffdee4dc4341e4a33fcfe884f28072, user_id=50dab514c64d47d095bd6d1ac8ce3b56, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.555728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45cc8d3-6a6a-4c2d-a0e9-3ac0fc58772c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.561608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1083/2143] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:57.565396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.577772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b6479-0aee-4acf-adaf-35fb0714640e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.583432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.584065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.584344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.584631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.585315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:26:57.585617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.587300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.609770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1084/2144] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:26:57.611606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.627802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1061/2145] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:57 2022] PUT /identity/v3/projects/cbffdee4dc4341e4a33fcfe884f28072/users/50dab514c64d47d095bd6d1ac8ce3b56/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:57.637453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005942b-076a-45df-b3ce-4adf6b380150 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.642803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270605-4564-49b3-837c-62675e9ae254 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.643383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270605-4564-49b3-837c-62675e9ae254 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.643758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30270605-4564-49b3-837c-62675e9ae254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.687902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-30270605-4564-49b3-837c-62675e9ae254 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:57.689211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30270605-4564-49b3-837c-62675e9ae254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.717949 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.718935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.719230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.719542 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.720321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.723514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.789673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1062/2146] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:57.802215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30270605-4564-49b3-837c-62675e9ae254 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.802985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:57.812714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30270605-4564-49b3-837c-62675e9ae254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:57.845187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1085/2147] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/projects => generated 371 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:26:57.847078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:57.905842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30270605-4564-49b3-837c-62675e9ae254 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.938828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d225163-fa91-42b8-94c1-4f00286d8882 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:57.943980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.944670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.944957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.946384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.947921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.950233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.968755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:57.969407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:57.969685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:57.969997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:57.981834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:57.983417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:57.993922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1063/2148] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:58.053519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1086/2149] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:26:57 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:26:58.057405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:58.067464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6da5b-8381-4d80-a3bc-2808992412a1 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:58.072176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:58.072772 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:58.073053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:58.073325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:58.073995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:58.075620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:58.086496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1064/2150] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:58.090606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:58.101462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf7ded-cc68-4197-93a6-dccf8d03f6a3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:58.106324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:58.106930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:58.107255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:58.107535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:58.108202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:58.109825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f53da3-f9ed-4712-a94d-c67f1f2f2cd7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:58.122562 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1087/2151] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:26:58.124485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:58.135274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41295f-b397-4c76-8452-4e79992dc676 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:58.140563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:58.141249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:58.141537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:58.141848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] PATH_INFO: `/v3/projects/86ad1eb80d2a42c9871f992a85b27a17/users/7c0b3d9264174863b78e100a659e90bf/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:58.162133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86ad1eb80d2a42c9871f992a85b27a17, user_id=7c0b3d9264174863b78e100a659e90bf, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:58.164820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:58.255744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1065/2152] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:26:58 2022] PUT /identity/v3/projects/86ad1eb80d2a42c9871f992a85b27a17/users/7c0b3d9264174863b78e100a659e90bf/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:26:58.260608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818072-9e59-4307-81a3-44f78d44e62d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:58.261881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818072-9e59-4307-81a3-44f78d44e62d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:58.261881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a818072-9e59-4307-81a3-44f78d44e62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:58.284561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6a818072-9e59-4307-81a3-44f78d44e62d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:58.286553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6a818072-9e59-4307-81a3-44f78d44e62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:58.353089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1088/2153] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:26:58.365159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:58.367880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:58.378297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:58.442631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04977697-8aa2-4e20-9791-832cdbea27cd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:58.447328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:58.448093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:58.448444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:58.449026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:58.462982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:58.470143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:58.475386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1066/2154] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:26:59.236609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:26:59.673125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67e373-7dec-43fe-9780-199aeb249c9a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:59.673610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67e373-7dec-43fe-9780-199aeb249c9a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:59.673746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67e373-7dec-43fe-9780-199aeb249c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:59.708469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4d67e373-7dec-43fe-9780-199aeb249c9a None None] MFA Rules not processed for user `50dab514c64d47d095bd6d1ac8ce3b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:26:59.709824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4d67e373-7dec-43fe-9780-199aeb249c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:59.788690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1089/2155] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:26:59.799913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:26:59.802212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:26:59.859966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1f998-45c4-4eec-9c44-755f4cf80e26 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:26:59.863835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:26:59.864459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:26:59.864740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:26:59.865011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:26:59.873264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:26:59.874728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:26:59.883567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1067/2156] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:26:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:00.402920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:00.430551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde261-d2ae-4d14-95ed-7ee88ad0cc31 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:00.430836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde261-d2ae-4d14-95ed-7ee88ad0cc31 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:00.431111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde261-d2ae-4d14-95ed-7ee88ad0cc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:00.460971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-86dde261-d2ae-4d14-95ed-7ee88ad0cc31 None None] MFA Rules not processed for user `7c0b3d9264174863b78e100a659e90bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:00.462233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-86dde261-d2ae-4d14-95ed-7ee88ad0cc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:00.529104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1090/2157] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:00.563399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:00.564793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:00.576915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af20f61-f174-49f5-bbb0-5fe59e4b4d0d service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:00.580513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:00.581100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:00.581373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:00.581640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:00.590058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:00.591467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5322a-77d7-40d3-852f-879e8f3bc9d7 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:00.600297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1068/2158] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:01.687992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:01.812334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:02.481037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:03.367611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:03.733983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:04.025210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:04.608163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:06.063700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:06.064243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:06.064335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:06.086009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] MFA Rules not processed for user `50dab514c64d47d095bd6d1ac8ce3b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:06.087217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:06.095247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1091/2159] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:06.669068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9253-db3b-435b-89e6-35aed35e9b4b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:06.669588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9253-db3b-435b-89e6-35aed35e9b4b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:06.669668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9253-db3b-435b-89e6-35aed35e9b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:06.692490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-983c9253-db3b-435b-89e6-35aed35e9b4b None None] MFA Rules not processed for user `7c0b3d9264174863b78e100a659e90bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:06.693797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-983c9253-db3b-435b-89e6-35aed35e9b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:06.726931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1069/2160] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:16.079867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=73 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:16.685367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=18 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:19.386367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:19.387913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:19.403377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc922b-28e7-4a4f-a6b6-c451acb5db7f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:19.410768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:19.411440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:19.411726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:19.412000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] PATH_INFO: `/v3/users/051d70cc75f140e6b7e1c855266014d8` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:19.413380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051d70cc75f140e6b7e1c855266014d8)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:19.415663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:19.668805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051d70cc75f140e6b7e1c855266014d8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:19.669497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051d70cc75f140e6b7e1c855266014d8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:19.679105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051d70cc75f140e6b7e1c855266014d8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:19.791198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051d70cc75f140e6b7e1c855266014d8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:19.792011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051d70cc75f140e6b7e1c855266014d8'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:19.828347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1092/2161] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:19 2022] DELETE /identity/v3/users/051d70cc75f140e6b7e1c855266014d8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:19.833006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:19.833507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:19.833676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:19.862205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:19.863085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:19.937333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1070/2162] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:19.950846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:19.952795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:20.024190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa4d12-5f10-47fb-b256-64be1912a8d4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:20.028163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:20.029082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:20.029082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:20.029329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:20.039150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:20.040672 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87867f29-f6da-4e85-9df5-bd3c89606d23 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:20.053080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1093/2163] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:27:20.592696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:20.594354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:20.648066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a80d1-e5da-4800-91cf-e1f079b139f9 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:20.664060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:20.665910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:20.667029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:20.667465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] PATH_INFO: `/v3/projects/228e69f3738f4c61a27dab62d5c3781d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:20.669012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228e69f3738f4c61a27dab62d5c3781d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:20.671791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:20.850817 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228e69f3738f4c61a27dab62d5c3781d', 'request_id': 'req-c358536c-cb5b-4e39-a167-ea0488efec41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:20.868320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228e69f3738f4c61a27dab62d5c3781d', 'request_id': 'req-c358536c-cb5b-4e39-a167-ea0488efec41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:20.963829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228e69f3738f4c61a27dab62d5c3781d', 'request_id': 'req-c358536c-cb5b-4e39-a167-ea0488efec41'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:20.982259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] The token cache is being invalidate because project 228e69f3738f4c61a27dab62d5c3781d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:20.982799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:20.983902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-c358536c-cb5b-4e39-a167-ea0488efec41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:20.987835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1071/2164] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:20 2022] DELETE /identity/v3/projects/228e69f3738f4c61a27dab62d5c3781d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:20.990827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:20.991322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:20.991665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.033249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:21.035901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:21.120586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1094/2165] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:21.128309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.128593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.128866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.154612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:21.156177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:21.168568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1072/2166] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:21.171681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.183018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab80e8-9a15-43a3-b44f-9010f6c669e5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.190573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.191279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.191632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.191997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.192753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:27:21.193118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.195001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.216523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1095/2167] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:27:21.217907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.228602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77299ea-d40a-4d47-ab81-577ca28647ae None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.239015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.239916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.240448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.241747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.241747 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.244816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.351495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1073/2168] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:21.361598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.367057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:21.376082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65791f-6400-4c2f-bf7b-41cad7602cf1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.472968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.474699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.474699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.474699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.474992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.477017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.590698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1096/2169] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:27:21 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:21.599927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.621436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373b4f2-71f3-4625-9b17-16fa3caec807 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.626342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.627017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.627296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.628161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.628883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.631052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.647625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1074/2170] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:21.654567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.664824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc2a26-7940-4157-8ad6-7d54839f8f73 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.672072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.672842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.673141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.673417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.674276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.676231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cb7f1-599a-4e35-9c00-e9d1c2cfeedc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.691167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1097/2171] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:21.695411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:21.711508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239469-2f3d-4ed6-8704-73bebe391cdc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:21.716699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:21.717357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.718142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.718443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] PATH_INFO: `/v3/projects/5836e2335d184b52a37d7effa572c5e4/users/65aa97150b6f437092f5bd5c90725880/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.738769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5836e2335d184b52a37d7effa572c5e4, user_id=65aa97150b6f437092f5bd5c90725880, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:21.741619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:21.820118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1075/2172] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:21 2022] PUT /identity/v3/projects/5836e2335d184b52a37d7effa572c5e4/users/65aa97150b6f437092f5bd5c90725880/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:21.830546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:21.830826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:21.831079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:21.863736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] MFA Rules not processed for user `65aa97150b6f437092f5bd5c90725880`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:21.864972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:21.971128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1098/2173] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:21.972444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.000881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:22.067283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b6d39-e228-4770-9d40-0887391f926a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.072479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.073763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.073763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.074145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.074875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.076999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:22.188388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1076/2174] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:22.192849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.209249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3276487-b150-415d-95c5-108b317fbf2f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.285382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.286035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.286313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.286585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.287291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.289223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:22.439140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:22.448810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1099/2175] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:27:22 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:22.450309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.465196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae2bd0-354e-45dc-a79b-b6a1a7912e29 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.470308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.470945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.471225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.471491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.472143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.473778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:22.490696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1077/2176] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:22.491844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.503642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b33de-5f5a-400a-902c-5e97c663adf0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.510224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.510822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.511115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.511388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.512042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.513677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d46ddf4-d54b-4792-85a6-5237f76b61e6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:22.527472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1100/2177] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:27:22.533559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.542170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b6df3-508a-4a98-b9b0-cd5499e0bbf0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.546776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.547453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.547731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.548003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] PATH_INFO: `/v3/projects/3706a5734bd04d67b2d9b71ba82cab75/users/12f6f82cbd0f42968d9eaf9a41468049/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.566151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3706a5734bd04d67b2d9b71ba82cab75, user_id=12f6f82cbd0f42968d9eaf9a41468049, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.574825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:22.675714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1078/2178] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:22 2022] PUT /identity/v3/projects/3706a5734bd04d67b2d9b71ba82cab75/users/12f6f82cbd0f42968d9eaf9a41468049/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:27:22.690328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.690845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.691256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.732608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] MFA Rules not processed for user `12f6f82cbd0f42968d9eaf9a41468049`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:22.733900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:22.823561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1101/2179] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:22.828081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:22.915383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a62212-c2b5-4687-8980-3fdb1a184e67 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:22.920465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:22.921202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:22.926568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:22.926568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:22.926568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:22.926568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.067814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1079/2180] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:27:22 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:23.075559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.084238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2440-f885-44b1-bda1-c7a28845571f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.162734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.163368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.163649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.163924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.164572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.167260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.287126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1102/2181] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:27:23 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:23.293211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.300116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae1f62-5ee9-459d-9e3e-1e6f668ccea8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.308168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.308358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.308687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.309093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.310052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.311821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.327255 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1080/2182] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:23.329442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.342318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15588a6c-1699-47f8-8bb8-7efc6adce081 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.347667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.348444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.348855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.349123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] PATH_INFO: `/v3/projects/fee9ac82b66f4298872095ebb6b5924b/users/0e67a96889c54772b597165f0b76e258/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.374394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee9ac82b66f4298872095ebb6b5924b, user_id=0e67a96889c54772b597165f0b76e258, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.377180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.472559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1103/2183] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:23 2022] PUT /identity/v3/projects/fee9ac82b66f4298872095ebb6b5924b/users/0e67a96889c54772b597165f0b76e258/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:23.476517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.487627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4320a5-f747-476b-915a-750a37485f39 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.537241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:23.553598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.554403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.554690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.555015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.555632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.557703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd610098-9073-45cf-914e-86f024386243 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.572381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1081/2184] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:23.573576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.584762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148343-2290-46af-9fda-5999abe1259a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.595980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.595980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.595980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.595980 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] PATH_INFO: `/v3/projects/fee9ac82b66f4298872095ebb6b5924b/users/0e67a96889c54772b597165f0b76e258/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.596507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee9ac82b66f4298872095ebb6b5924b, user_id=0e67a96889c54772b597165f0b76e258, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.596507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.608791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1104/2185] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:23 2022] PUT /identity/v3/projects/fee9ac82b66f4298872095ebb6b5924b/users/0e67a96889c54772b597165f0b76e258/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:23.619738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.619978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.620180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.650685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] MFA Rules not processed for user `0e67a96889c54772b597165f0b76e258`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:23.652230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:23.734555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1082/2186] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:23.746680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:23.748380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:23.786837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:23.815155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916bfea-3f06-4f6a-b8e6-bdf48dd7741d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:23.820383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:23.820999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:23.821430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:23.821812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:23.908800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:23.910279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:23.923097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1105/2187] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:27.108891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:27.118516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936a500-4558-4757-9f3d-c85157536a78 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:27.123180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:27.123777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:27.124072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:27.124325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:27.182637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:27.184237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:27.193213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1083/2188] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:29.084602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:29.093554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b859ab7-c4dd-48f3-8c19-fc26b165e66d service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:29.097338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:29.097979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:29.098259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:29.098532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:29.108872 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:29.110344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:29.120557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1106/2189] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:32.712939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:32.714476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:32.802908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca231ee-bacc-4d08-bc1f-3987a02e76e2 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:32.808597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:32.809198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:32.809548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:32.809851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:32.811597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:32.824543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:32.828760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:32.830399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20527220-9a68-48a4-881c-fe7a3ab1cfbd service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:32.840135 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1084/2190] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:36.246909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:36.320003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b538a2-4b96-471c-b48d-71505ff998e5 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:36.325527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:36.326266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:36.326544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:36.326818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] PATH_INFO: `/v3/users/65aa97150b6f437092f5bd5c90725880` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:36.327959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65aa97150b6f437092f5bd5c90725880)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:36.329910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:36.556314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65aa97150b6f437092f5bd5c90725880'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:36.556895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65aa97150b6f437092f5bd5c90725880'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:36.565486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65aa97150b6f437092f5bd5c90725880'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:36.654971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65aa97150b6f437092f5bd5c90725880'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:36.660596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65aa97150b6f437092f5bd5c90725880'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:36.664455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:36.675440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1107/2191] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:36 2022] DELETE /identity/v3/users/65aa97150b6f437092f5bd5c90725880 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:36.680150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:36.680666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:36.680666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:36.701307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:36.702530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:36.768031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1085/2192] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:36.778684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:36.780184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:36.858174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f0edb-5699-4b0a-9a49-e25548a10e56 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:36.861880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:36.862563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:36.862837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:36.863123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:36.877440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:36.881064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:36.900707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1108/2193] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:37.407773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:37.409126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:37.425118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280bd06-0113-40c9-b2b4-7da105c352cd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:37.430536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:37.431207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:37.431548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:37.431787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] PATH_INFO: `/v3/projects/5836e2335d184b52a37d7effa572c5e4` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:37.433115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5836e2335d184b52a37d7effa572c5e4)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:37.435166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:37.606965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5836e2335d184b52a37d7effa572c5e4', 'request_id': 'req-4c613681-6b52-4838-8518-45feae5e180a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.613511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5836e2335d184b52a37d7effa572c5e4', 'request_id': 'req-4c613681-6b52-4838-8518-45feae5e180a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.688755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5836e2335d184b52a37d7effa572c5e4', 'request_id': 'req-4c613681-6b52-4838-8518-45feae5e180a'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.701596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] The token cache is being invalidate because project 5836e2335d184b52a37d7effa572c5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:37.701918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.702573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.704179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1086/2194] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:37 2022] DELETE /identity/v3/projects/5836e2335d184b52a37d7effa572c5e4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:37.708212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:37.719006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedcd69-dbdc-4ff7-ae91-e3c0eac132a0 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:37.777400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:37.778060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:37.778330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:37.778656 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] PATH_INFO: `/v3/users/12f6f82cbd0f42968d9eaf9a41468049` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:37.779914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f6f82cbd0f42968d9eaf9a41468049)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:37.781875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:37.961324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f6f82cbd0f42968d9eaf9a41468049'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.961971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f6f82cbd0f42968d9eaf9a41468049'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:37.969267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f6f82cbd0f42968d9eaf9a41468049'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.065783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f6f82cbd0f42968d9eaf9a41468049'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.066418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f6f82cbd0f42968d9eaf9a41468049'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.084601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1109/2195] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:37 2022] DELETE /identity/v3/users/12f6f82cbd0f42968d9eaf9a41468049 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:38.536749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:38.538354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:38.552218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c613681-6b52-4838-8518-45feae5e180a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:38.613675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:38.614352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:38.614625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:38.614890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] PATH_INFO: `/v3/projects/3706a5734bd04d67b2d9b71ba82cab75` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:38.616034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3706a5734bd04d67b2d9b71ba82cab75)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:38.618061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:38.639683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:38.739550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3706a5734bd04d67b2d9b71ba82cab75', 'request_id': 'req-93764542-bff9-450f-bc69-c465ec6b9e5e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.745791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3706a5734bd04d67b2d9b71ba82cab75', 'request_id': 'req-93764542-bff9-450f-bc69-c465ec6b9e5e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.803392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3706a5734bd04d67b2d9b71ba82cab75', 'request_id': 'req-93764542-bff9-450f-bc69-c465ec6b9e5e'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.814413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] The token cache is being invalidate because project 3706a5734bd04d67b2d9b71ba82cab75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:38.814702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.815630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:38.817131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1087/2196] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:38 2022] DELETE /identity/v3/projects/3706a5734bd04d67b2d9b71ba82cab75 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:27:38.821154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:38.822630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:38.837214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c6e8-a56d-4170-97c6-cc216e70d1e4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:38.849044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:38.899749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:38.900397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:38.900704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:38.900982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] PATH_INFO: `/v3/users/0e67a96889c54772b597165f0b76e258` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:38.902724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e67a96889c54772b597165f0b76e258)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:38.904677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:39.109085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e67a96889c54772b597165f0b76e258'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:39.109674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e67a96889c54772b597165f0b76e258'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:39.117368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e67a96889c54772b597165f0b76e258'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:39.203747 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e67a96889c54772b597165f0b76e258'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:39.204615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4a4645a5-7fd6-4f9f-b8a6-ec2cce970c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e67a96889c54772b597165f0b76e258'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:39.223132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1110/2197] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:38 2022] DELETE /identity/v3/users/0e67a96889c54772b597165f0b76e258 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:39.863646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:39.865384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:39.881440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764542-bff9-450f-bc69-c465ec6b9e5e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:40.004699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:40.005675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.006047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.006533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] PATH_INFO: `/v3/projects/fee9ac82b66f4298872095ebb6b5924b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.007868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee9ac82b66f4298872095ebb6b5924b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:40.010626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:40.182944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee9ac82b66f4298872095ebb6b5924b', 'request_id': 'req-e56f8206-e1be-4100-8437-2c0d05d04d3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:40.189948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee9ac82b66f4298872095ebb6b5924b', 'request_id': 'req-e56f8206-e1be-4100-8437-2c0d05d04d3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:40.295875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee9ac82b66f4298872095ebb6b5924b', 'request_id': 'req-e56f8206-e1be-4100-8437-2c0d05d04d3f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:40.307601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] The token cache is being invalidate because project fee9ac82b66f4298872095ebb6b5924b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:40.308110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:40.308952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e56f8206-e1be-4100-8437-2c0d05d04d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:40.311079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1088/2198] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:39 2022] DELETE /identity/v3/projects/fee9ac82b66f4298872095ebb6b5924b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:40.316257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.316757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.316854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.337401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:40.338671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:40.404329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1111/2199] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:40.412438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.412738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.413007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.433695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:40.435082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:40.443316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1089/2200] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:40.447680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:40.458230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cd86c-0ca1-48e0-976c-fd0af7fa0e59 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:40.462959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:40.463560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.463833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.464096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.464746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:27:40.465044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:40.467254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:40.480368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1112/2201] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:27:40.484978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:40.495091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08025e-d209-41db-aee3-8fd8d02fda48 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:40.501248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:40.501895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.502176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.502444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.503089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:40.504995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:40.704049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1090/2202] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:27:40 2022] POST /identity/v3/projects => generated 369 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:40.715065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:40.726748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ce0b7-0823-4e6e-81ab-eef78d35b662 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:40.744662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:40.792716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:40.793329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.793636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.794618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.795256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:40.797186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:40.953271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1113/2203] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:27:40 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:27:40.957469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:40.969401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b873f0e-bccc-41ba-80c2-61304b5d13aa admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:40.974560 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:40.975179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:40.975448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:40.975719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:40.976454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:40.978097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:40.980292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:40.989005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1091/2204] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:27:40.995023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:41.008155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693726-47ce-4fc5-9b57-fd447aa31bb3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:41.013989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:41.014388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.014679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.015367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.016476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:41.019308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3139f8-8778-43c1-8f90-31a691fe4cc9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:41.035887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1114/2205] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:41.037082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:41.048118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeee3ee-fe2a-4554-9e59-1db56fec3ea9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:41.052716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:41.054513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.054513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.054513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] PATH_INFO: `/v3/projects/b7b159ed637c46c3afa968fa74af7afc/users/16666e5d2607498090cce8e7afe561e3/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.071418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b159ed637c46c3afa968fa74af7afc, user_id=16666e5d2607498090cce8e7afe561e3, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:41.074241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:41.171847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1092/2206] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:41 2022] PUT /identity/v3/projects/b7b159ed637c46c3afa968fa74af7afc/users/16666e5d2607498090cce8e7afe561e3/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:41.186457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.186739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.187018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.219168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] MFA Rules not processed for user `16666e5d2607498090cce8e7afe561e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:41.220464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:41.296445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1115/2207] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:41.307603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:41.311417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:41.377300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0459f1-b727-472f-bfba-33987c37de71 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:41.381154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:41.381802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.382160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.382341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.392024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:41.393794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad442f1-16e8-4132-b593-1902b10fb3d8 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:41.403291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1093/2208] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:41.466017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:41.547651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea730-4db7-4d09-bc28-cc0005aa538a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:41.552248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:41.552892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.553160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.553429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] PATH_INFO: `/v3/users/16666e5d2607498090cce8e7afe561e3` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.554619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16666e5d2607498090cce8e7afe561e3)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:41.556560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:41.743855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:41.758764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16666e5d2607498090cce8e7afe561e3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:41.759417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16666e5d2607498090cce8e7afe561e3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:41.771306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16666e5d2607498090cce8e7afe561e3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:41.866704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16666e5d2607498090cce8e7afe561e3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:41.867332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16666e5d2607498090cce8e7afe561e3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:41.891737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1116/2209] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:41 2022] DELETE /identity/v3/users/16666e5d2607498090cce8e7afe561e3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:41.893764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:41.894059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:41.894340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:41.916778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:41.918180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:41.988688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1094/2210] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:41.998398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:41.999861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:42.069897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cd157-fd76-44f5-8067-d0c804d3fb31 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:42.074163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:42.075367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:42.075643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:42.079266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:42.086061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:42.087517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328178e-51b2-4ea1-85cb-e258ec7675cf service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:42.098887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1117/2211] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:42.743208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:42.744681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:42.760266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50ef35-d877-4041-a29e-66df4028c97f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:42.767476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:42.768488 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:42.769040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:42.769330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] PATH_INFO: `/v3/projects/b7b159ed637c46c3afa968fa74af7afc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:42.770996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b159ed637c46c3afa968fa74af7afc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:42.772922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:42.914148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b159ed637c46c3afa968fa74af7afc', 'request_id': 'req-e92bf76f-c470-4405-85f3-379cecea05c6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:42.920999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b159ed637c46c3afa968fa74af7afc', 'request_id': 'req-e92bf76f-c470-4405-85f3-379cecea05c6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:42.975426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b159ed637c46c3afa968fa74af7afc', 'request_id': 'req-e92bf76f-c470-4405-85f3-379cecea05c6'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:42.984067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:42.988310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] The token cache is being invalidate because project b7b159ed637c46c3afa968fa74af7afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:42.988603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:42.989244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e92bf76f-c470-4405-85f3-379cecea05c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:42.990802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1095/2212] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:42 2022] DELETE /identity/v3/projects/b7b159ed637c46c3afa968fa74af7afc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:42.995795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3555de-604e-4231-a67c-3b014bce235c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:42.996227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3555de-604e-4231-a67c-3b014bce235c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:42.996355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3555de-604e-4231-a67c-3b014bce235c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.022192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4c3555de-604e-4231-a67c-3b014bce235c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:43.023434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4c3555de-604e-4231-a67c-3b014bce235c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:43.048298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b88f-412f-447e-9414-608e037f03fa None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.048679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b88f-412f-447e-9414-608e037f03fa None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.048953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b88f-412f-447e-9414-608e037f03fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.068859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-a955b88f-412f-447e-9414-608e037f03fa None None] MFA Rules not processed for user `7c0b3d9264174863b78e100a659e90bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:43.070245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a955b88f-412f-447e-9414-608e037f03fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:43.098850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1118/2213] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:43.108025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.108495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.108791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.132151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:43.133435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:43.139421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1096/2214] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:43.147983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1119/2215] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:27:43.152350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955b88f-412f-447e-9414-608e037f03fa None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.163927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955b88f-412f-447e-9414-608e037f03fa None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.168746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.170027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.170366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.170640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.171359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:27:43.171667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.173246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.186033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1097/2216] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:27:43.190815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.203714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fa1fb-5445-4226-8d64-48468e5b03d2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.208809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.209608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.209955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.210185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.210824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.212752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.277433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1120/2217] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:27:43 2022] POST /identity/v3/projects => generated 399 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:43.279212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.289929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b07dc-edd2-4b97-bba2-be127972b5df admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.352930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.353541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.353920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.354136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.359721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.359721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.462338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1098/2218] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:27:43 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:27:43.467116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.478964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eacd20-c751-4973-9c46-718ab542d8cd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.485489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.485489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.485489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.485489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.486122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.487915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.501690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1121/2219] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:43.502956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.515838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6f617-2b47-422e-8bc9-a9a56ba1d2fd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.520953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.521558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.521864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.522132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.522846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.524468 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e082d-af38-4b08-aee7-4ca3a6342efc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.536334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1099/2220] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:27:43.540587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.551006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d85277-68bb-403a-9497-5e23e71b3ae7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.555603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.556250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.556526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.556795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] PATH_INFO: `/v3/projects/031d3ce853334c7c9db03338ae5f35cd/users/4f325deeedeb4b42b1b4a05e7f480989/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.567773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:43.576085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d3ce853334c7c9db03338ae5f35cd, user_id=4f325deeedeb4b42b1b4a05e7f480989, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.580183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.662437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1122/2221] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:27:43 2022] PUT /identity/v3/projects/031d3ce853334c7c9db03338ae5f35cd/users/4f325deeedeb4b42b1b4a05e7f480989/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:27:43.674193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.674481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.674760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.703295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] MFA Rules not processed for user `4f325deeedeb4b42b1b4a05e7f480989`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:43.704503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:43.771824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:43.780210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1100/2222] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:43.798867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:43.812412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:43.896166 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e401cb6-4549-4171-a0ea-3db628816be6 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:43.911311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:43.912048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:43.912420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:43.912789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:43.923603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:43.924868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df96e0-03a1-4cb3-9fe4-4863bf6b4d6f service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:43.935067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1123/2223] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:43 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:53.929993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:57.924029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:57.925137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:58.000074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30704e1a-cb6a-4ffe-a813-c79bf887672e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:58.009402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:58.010100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:58.012354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:58.012354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] PATH_INFO: `/v3/users/7b8c60a410394867a333f0febb06cda8` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:58.012354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8c60a410394867a333f0febb06cda8)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:58.013733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:58.178568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8c60a410394867a333f0febb06cda8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:58.179159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8c60a410394867a333f0febb06cda8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:58.261622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8c60a410394867a333f0febb06cda8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:58.272275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8c60a410394867a333f0febb06cda8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:58.285969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8c60a410394867a333f0febb06cda8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:58.297840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1101/2224] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:57 2022] DELETE /identity/v3/users/7b8c60a410394867a333f0febb06cda8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:58.302579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:58.303064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:58.303205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:58.324843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:27:58.326092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:58.388739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1124/2225] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:27:58.400362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:58.401807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:58.469784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac313dc-5acb-47e5-a00b-33efcb9e4d77 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:58.473561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:58.474181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:58.474457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:58.474725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:58.485000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:58.486456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:58.496124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1102/2226] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:27:58.941312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:58.942966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:58.967429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4c520-e128-4209-a22d-80d78eec7f3a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:58.973511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:58.973954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:58.974315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:58.974772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] PATH_INFO: `/v3/projects/a12ed1b1ebff4fb3a1522cc79fe54e49` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:58.976325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12ed1b1ebff4fb3a1522cc79fe54e49)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:58.978579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:59.148307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12ed1b1ebff4fb3a1522cc79fe54e49', 'request_id': 'req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.154835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12ed1b1ebff4fb3a1522cc79fe54e49', 'request_id': 'req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.244785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12ed1b1ebff4fb3a1522cc79fe54e49', 'request_id': 'req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.268337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] The token cache is being invalidate because project a12ed1b1ebff4fb3a1522cc79fe54e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:27:59.269069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.270217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-62e8abb8-58f4-486a-afa9-5b66f63f6a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.272478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1125/2227] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:27:58 2022] DELETE /identity/v3/projects/a12ed1b1ebff4fb3a1522cc79fe54e49 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:27:59.699181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:27:59.700651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:27:59.714264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19eb8-7dd9-494c-89e5-3b5ed560f6b7 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:27:59.732535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:27:59.773767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:27:59.774482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:27:59.774964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:27:59.775491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] PATH_INFO: `/v3/users/4f325deeedeb4b42b1b4a05e7f480989` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:27:59.777047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f325deeedeb4b42b1b4a05e7f480989)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:27:59.779039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:27:59.990371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f325deeedeb4b42b1b4a05e7f480989'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:27:59.991046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f325deeedeb4b42b1b4a05e7f480989'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:00.067504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f325deeedeb4b42b1b4a05e7f480989'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:00.075272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f325deeedeb4b42b1b4a05e7f480989'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:00.087273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f325deeedeb4b42b1b4a05e7f480989'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:00.093197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1103/2228] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:27:59 2022] DELETE /identity/v3/users/4f325deeedeb4b42b1b4a05e7f480989 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:28:00.098551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:00.099056 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:00.099184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:00.120210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:00.121430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:00.189012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1126/2229] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:00.199236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:00.200957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:00.275876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5cce1-c61a-4447-bd77-5712cbf0e3bd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:00.279535 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:00.280141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:00.280417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:00.280686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:00.290952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:00.292413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd08c2f-3973-4680-a0b3-a5ff519cf15e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:00.304343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1104/2230] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:28:00.801345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:00.802886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:00.815597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251a6e3-5151-4029-96a6-6efa5e4688ab None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:00.820389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:00.821035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:00.821315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:00.822017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] PATH_INFO: `/v3/projects/031d3ce853334c7c9db03338ae5f35cd` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:00.822825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] RBAC: Authorizing `identity:delete_project(project_id=031d3ce853334c7c9db03338ae5f35cd)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:00.824724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:00.848870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:00.956084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031d3ce853334c7c9db03338ae5f35cd', 'request_id': 'req-39c947d2-15a8-48b0-b72b-62dded89910b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:00.962251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031d3ce853334c7c9db03338ae5f35cd', 'request_id': 'req-39c947d2-15a8-48b0-b72b-62dded89910b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:01.079652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031d3ce853334c7c9db03338ae5f35cd', 'request_id': 'req-39c947d2-15a8-48b0-b72b-62dded89910b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:01.090844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] The token cache is being invalidate because project 031d3ce853334c7c9db03338ae5f35cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:28:01.091128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:01.091765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:01.093264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1127/2231] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:28:00 2022] DELETE /identity/v3/projects/031d3ce853334c7c9db03338ae5f35cd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:28:10.302873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:11.273381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:22.820684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2173-47cc-4978-afa7-57dea86eb18f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:22.821386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2173-47cc-4978-afa7-57dea86eb18f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:22.821386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2173-47cc-4978-afa7-57dea86eb18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:22.844094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2d5f2173-47cc-4978-afa7-57dea86eb18f None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:22.845960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2d5f2173-47cc-4978-afa7-57dea86eb18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:22.917403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1105/2232] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:28:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:22.968671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:22.970652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:23.068403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c947d2-15a8-48b0-b72b-62dded89910b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:23.075534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:23.076223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:23.076635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:23.077081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:23.086778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:23.088491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:23.099605 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1128/2233] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:28:32.118339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb895a6-4807-4bb0-a04b-cc8e1c25ae15 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:32.119829 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb895a6-4807-4bb0-a04b-cc8e1c25ae15 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:32.120551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb895a6-4807-4bb0-a04b-cc8e1c25ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:32.154077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8bb895a6-4807-4bb0-a04b-cc8e1c25ae15 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:32.155899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8bb895a6-4807-4bb0-a04b-cc8e1c25ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:32.173637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1106/2234] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:28:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:32.199288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:32.218899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451bea6-1e24-4443-b860-2423be5ef551 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:32.227257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:32.228221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:32.228593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:32.229062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:32.244778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:32.247294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:32.260356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1129/2235] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:28:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:28:42.144245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=28 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:42.259099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=38 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:46.999753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:47.003101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:47.003101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:47.020766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:47.025854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:47.029633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1107/2236] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:47.042713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:47.054641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55a8db-4b8a-42ab-84f8-5fa0b3254e48 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:47.059727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:47.060330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:47.060606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:47.060876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:47.070760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:47.072391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:47.081041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1130/2237] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:28:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 21:28:53.763961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:53.765794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:53.845499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602fd02-4e3c-44c6-8904-d9f258999174 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:53.850356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:53.850987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:53.851263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:53.851531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] PATH_INFO: `/v3/users/7c0b3d9264174863b78e100a659e90bf` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:53.852774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0b3d9264174863b78e100a659e90bf)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:53.854903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:54.066838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0b3d9264174863b78e100a659e90bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.068075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0b3d9264174863b78e100a659e90bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.155288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0b3d9264174863b78e100a659e90bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.163634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0b3d9264174863b78e100a659e90bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.175608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9c018b6a-e40f-4873-8418-c4331efa2b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0b3d9264174863b78e100a659e90bf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.182749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1108/2238] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:28:53 2022] DELETE /identity/v3/users/7c0b3d9264174863b78e100a659e90bf => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:28:54.433361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:54.444903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207e73-f3b4-4975-a7dc-f05d28f2d832 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:54.522500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:54.523200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:54.525024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:54.525024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] PATH_INFO: `/v3/projects/86ad1eb80d2a42c9871f992a85b27a17` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:54.525024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86ad1eb80d2a42c9871f992a85b27a17)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:54.527084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:54.713409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ad1eb80d2a42c9871f992a85b27a17', 'request_id': 'req-850a501b-9b17-4c41-8aac-abdf9a440c24'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.721441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ad1eb80d2a42c9871f992a85b27a17', 'request_id': 'req-850a501b-9b17-4c41-8aac-abdf9a440c24'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.979941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ad1eb80d2a42c9871f992a85b27a17', 'request_id': 'req-850a501b-9b17-4c41-8aac-abdf9a440c24'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.992760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] The token cache is being invalidate because project 86ad1eb80d2a42c9871f992a85b27a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:28:54.993773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.994020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-850a501b-9b17-4c41-8aac-abdf9a440c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:28:54.999480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1131/2239] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:28:54 2022] DELETE /identity/v3/projects/86ad1eb80d2a42c9871f992a85b27a17 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:28:55.006431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.006951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.007062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.030957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:55.032457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:55.098512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1109/2240] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:55.107072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.107351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.107627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.134334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:55.135593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:55.144934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1132/2241] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:55.154851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.168265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc18d6f-d9cc-4120-ba32-259516c58108 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.174143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.174857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.175039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.175402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.175966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:28:55.176306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.178587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.188409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:55.192815 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1110/2242] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:28:55.197318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.207586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81973a96-9955-476f-95c1-2ac15a2d28f5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.214866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.224900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.224900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.224900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.224900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.224900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.334512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1133/2243] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:28:55 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:28:55.338743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.349678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd566319-e8b2-4303-b95b-bcef9571cc1a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.413087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.413736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.414054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.414328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.414962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.416895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.568453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1111/2244] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:28:55 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:28:55.572577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.584611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91306503-9752-4f67-a453-51702ef6a0f0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.594214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.594214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.594214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.594214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.594682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.595022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.609903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1134/2245] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:28:55.611140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.623319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a0bab-3219-49a1-a1c5-56c898949a23 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.628525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.629183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.629503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.629852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.630679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.632492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db53fe-d6cd-418f-9c5d-5d8f0cb2c752 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.644814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1112/2246] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:28:55.649739 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.661941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a0f8-dcb4-4133-a303-52954ee9cc9d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.667543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.668205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.668479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.668746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] PATH_INFO: `/v3/projects/99ed0de2b2bf4173919e3ed69467f408/users/f7f0dd60f95b442aa893f1a1eda4e275/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.687172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ed0de2b2bf4173919e3ed69467f408, user_id=f7f0dd60f95b442aa893f1a1eda4e275, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.689602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.755969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1135/2247] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:28:55 2022] PUT /identity/v3/projects/99ed0de2b2bf4173919e3ed69467f408/users/f7f0dd60f95b442aa893f1a1eda4e275/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:28:55.766821 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dac31-d457-4924-90a7-216b5b4eb150 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.767105 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dac31-d457-4924-90a7-216b5b4eb150 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.767378 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dac31-d457-4924-90a7-216b5b4eb150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.803434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bd5dac31-d457-4924-90a7-216b5b4eb150 None None] MFA Rules not processed for user `f7f0dd60f95b442aa893f1a1eda4e275`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:55.804669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bd5dac31-d457-4924-90a7-216b5b4eb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:55.882207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1113/2248] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:55.893545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:55.895901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:55.912141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:55.959102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419a17a-414d-4c78-a5d7-0dba90420a04 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:55.963799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:55.964610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:55.967764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:55.967764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:55.976292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:55.977795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b96787-dbb2-4284-861a-844210540300 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:55.988022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1136/2249] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:28:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:28:58.451062 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:58.453547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:58.453547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] PATH_INFO: `/` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:58.453932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1114/2250] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:28:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:28:58.461732 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1eedc-e0f1-44b1-9129-73ebfe35220b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:58.462636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1eedc-e0f1-44b1-9129-73ebfe35220b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:58.462916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1eedc-e0f1-44b1-9129-73ebfe35220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:58.483704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-eca1eedc-e0f1-44b1-9129-73ebfe35220b None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:58.485475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-eca1eedc-e0f1-44b1-9129-73ebfe35220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:58.557698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1137/2251] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:28:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:58.570637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:58.572727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:58.646671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e45df-bc1a-4677-984e-1110bfafc34e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:58.650555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:58.651159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:58.651439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:58.651712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:58.662392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:58.663820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb87e87-123d-4ea8-9387-03f248d8bcdf service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:58.674492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1115/2252] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:28:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:28:58.963022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150e43-ecbc-4031-9f02-04d212c19ad1 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:58.963217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150e43-ecbc-4031-9f02-04d212c19ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:58.963457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43150e43-ecbc-4031-9f02-04d212c19ad1 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:58.964924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1138/2253] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:28:58.970558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:58.970846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:58.971122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:58.994017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] MFA Rules not processed for user `e1564eaa8d8543faa415c09ddbdf4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:58.995340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:59.002030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:28:59.043197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9368ec-4d47-4fda-8b93-278728871046 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:59.043480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9368ec-4d47-4fda-8b93-278728871046 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:59.043755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9368ec-4d47-4fda-8b93-278728871046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:59.059909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1116/2254] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:28:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:59.063870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8c9368ec-4d47-4fda-8b93-278728871046 None None] MFA Rules not processed for user `f7f0dd60f95b442aa893f1a1eda4e275`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:59.065087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8c9368ec-4d47-4fda-8b93-278728871046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:59.072187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1139/2255] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:28:59.072375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:59.074115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:59.085742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3c7fc-5c6a-4ac9-82c3-64f82d138c1d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:59.089538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:59.090114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:59.090393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:59.090666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:59.098230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:59.100209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d61a1-c7b7-4037-a102-401bf110a011 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:59.108531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1117/2256] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:28:59.203336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dffb8f-7023-4397-bcf4-8392bc69c306 None None] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:59.203623 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dffb8f-7023-4397-bcf4-8392bc69c306 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:59.203903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dffb8f-7023-4397-bcf4-8392bc69c306 None None] PATH_INFO: `/` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:59.205409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1140/2257] 10.222.0.99 () {54 vars in 921 bytes} [Fri Sep 2 21:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 21:28:59.212147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4de5b4-1046-4505-951e-bb028758500e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:59.212516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4de5b4-1046-4505-951e-bb028758500e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:59.212847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4de5b4-1046-4505-951e-bb028758500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:59.240935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ac4de5b4-1046-4505-951e-bb028758500e None None] MFA Rules not processed for user `2d95be82e9e04470902aef0d9bab1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:28:59.242185 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ac4de5b4-1046-4505-951e-bb028758500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:28:59.252498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1118/2258] 10.222.0.99 () {58 vars in 1018 bytes} [Fri Sep 2 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:28:59.263657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dffb8f-7023-4397-bcf4-8392bc69c306 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:28:59.277940 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dffb8f-7023-4397-bcf4-8392bc69c306 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:28:59.281968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:28:59.282670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:28:59.283028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:28:59.283375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:28:59.295674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:28:59.297254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:28:59.310898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1141/2259] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:29:06.673982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:06.673982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:06.674436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:06.694911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:06.696224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:06.757621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1119/2260] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:29:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:06.769253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:06.771372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:06.849228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af3069-1248-45bf-ac60-8b656b0e6a8e service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:06.855113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:06.855723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:06.856002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:06.858540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:06.865312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:06.867516 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:06.876201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1142/2261] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:16.750676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=52 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:16.935698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:21.600276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:21.601708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:21.614058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5905d-0264-4719-8213-6b3cf4ba4399 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:21.617708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:21.618350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:21.618635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:21.618906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:21.620691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:21.637961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:21.639431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26463ef0-789b-4bb5-b0d8-0db39b159816 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:21.652547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1120/2262] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:29:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:29:22.270241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:22.281339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ecd3-2558-4c46-9116-e4e9213387a2 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:22.285020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:22.285646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:22.285959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:22.286235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:22.287978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:22.301831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:22.303292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:22.314914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1143/2263] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:29:22.975665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a924-1bf4-43fe-be45-dddbd95d7453 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:22.976189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a924-1bf4-43fe-be45-dddbd95d7453 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:22.976189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64a924-1bf4-43fe-be45-dddbd95d7453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:22.996602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-eb64a924-1bf4-43fe-be45-dddbd95d7453 None None] MFA Rules not processed for user `50dab514c64d47d095bd6d1ac8ce3b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:22.997911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-eb64a924-1bf4-43fe-be45-dddbd95d7453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:23.054337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1121/2264] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:29:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:32.310014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=6 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:33.047349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=31 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:34.707939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:34.709444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:34.726094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da9461-24f9-4950-a46f-4369ee5cb1b0 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:34.732647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:34.733353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:34.733691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:34.734103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:34.735880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:34.802777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:34.804363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:34.814802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1144/2265] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:29:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:29:43.996655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f7dc-ca41-4ddf-a3dd-30b06d71353d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:43.997348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f7dc-ca41-4ddf-a3dd-30b06d71353d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:43.997348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8f7dc-ca41-4ddf-a3dd-30b06d71353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:44.020604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-55f8f7dc-ca41-4ddf-a3dd-30b06d71353d None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:44.020604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-55f8f7dc-ca41-4ddf-a3dd-30b06d71353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:44.029887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1122/2266] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:29:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:29:44.055313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:44.068882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107209e3-e89e-4f02-89d9-5e1fdf7ad8c8 service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:44.075836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:44.076580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:44.076927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:44.077270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:44.088176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:44.089830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:44.100719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1145/2267] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:47.200108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760bef0-470d-4e31-b087-d4fe16ce1dbb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:47.200559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760bef0-470d-4e31-b087-d4fe16ce1dbb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:47.200680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760bef0-470d-4e31-b087-d4fe16ce1dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:47.219824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b760bef0-470d-4e31-b087-d4fe16ce1dbb None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:47.221054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b760bef0-470d-4e31-b087-d4fe16ce1dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:47.229482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1123/2268] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:47.242620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:47.255142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c58d5e-580f-4bce-8b63-df3fb233738b service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:47.260652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:47.261266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:47.261549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:47.261922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:47.271498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:47.273108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:47.282913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1146/2269] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:48.442706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998760-3bd1-4292-84c2-993665901c24 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:48.443220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998760-3bd1-4292-84c2-993665901c24 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:48.443315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998760-3bd1-4292-84c2-993665901c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:48.468154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c0998760-3bd1-4292-84c2-993665901c24 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:48.468154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c0998760-3bd1-4292-84c2-993665901c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:48.476357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1124/2270] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:29:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:48.492376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:48.504934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285215f-8d87-4fbf-ab35-6c78f57ed751 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:48.511266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:48.512081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:48.512398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:48.512736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:48.521610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:48.523226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:48.533074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1147/2271] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:49.320219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377466-d692-48ee-82b0-0004f719a1df None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:49.320494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377466-d692-48ee-82b0-0004f719a1df None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:49.320770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377466-d692-48ee-82b0-0004f719a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:49.347460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-18377466-d692-48ee-82b0-0004f719a1df None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:49.348898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-18377466-d692-48ee-82b0-0004f719a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:49.357054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1125/2272] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:49.375062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:49.386045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e90d1-0030-4072-a530-fdb648e57144 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:49.391713 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:49.392360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:49.392639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:49.392912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:49.403349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:49.405192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:49.413862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1148/2273] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:50.053327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:50.053753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:50.053902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:50.072947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:50.074695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:50.082084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1126/2274] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:50.095014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:50.106253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f312b-f3ff-43cc-b95d-ac111774d330 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:50.111537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:50.112186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:50.112462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:50.112733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:50.122438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:50.123762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b5f56-f3c9-4b09-9f3c-d52b64fed99d service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:50.138761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1149/2275] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:29:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:29:53.131865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:53.199622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b1774-86f6-4ef7-b47d-65503f98a793 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:53.204295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:53.204945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:53.205217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:53.205487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] PATH_INFO: `/v3/users/f7f0dd60f95b442aa893f1a1eda4e275` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:53.206714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f0dd60f95b442aa893f1a1eda4e275)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:53.208666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:53.425290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f0dd60f95b442aa893f1a1eda4e275'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:53.425924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f0dd60f95b442aa893f1a1eda4e275'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:53.528037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f0dd60f95b442aa893f1a1eda4e275'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:53.536411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f0dd60f95b442aa893f1a1eda4e275'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:53.550482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f0dd60f95b442aa893f1a1eda4e275'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:53.556803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1127/2276] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:29:53 2022] DELETE /identity/v3/users/f7f0dd60f95b442aa893f1a1eda4e275 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:29:53.561391 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:53.561902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:53.562003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:53.583145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:29:53.584364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:53.651939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1150/2277] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:29:53.662291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:53.663734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:53.717682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:53.723017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69b3b5-dd57-4179-ba9c-fd72ab21c942 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:53.726876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:53.730945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:53.730945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:53.730945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:53.737853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:53.739540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:53.749202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1128/2278] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:29:54.016021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:54.017480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:54.031679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3213c-5de6-4df8-861e-2f1173ee5206 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:54.039310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:54.040412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:54.041211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:54.041743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] PATH_INFO: `/v3/projects/99ed0de2b2bf4173919e3ed69467f408` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:54.043722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ed0de2b2bf4173919e3ed69467f408)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:54.048143 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:54.142500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:29:54.157694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ed0de2b2bf4173919e3ed69467f408', 'request_id': 'req-04abd09f-4ccb-46e4-9fe1-4056d4c41600'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:54.163880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ed0de2b2bf4173919e3ed69467f408', 'request_id': 'req-04abd09f-4ccb-46e4-9fe1-4056d4c41600'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:54.213455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ed0de2b2bf4173919e3ed69467f408', 'request_id': 'req-04abd09f-4ccb-46e4-9fe1-4056d4c41600'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:54.226037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] The token cache is being invalidate because project 99ed0de2b2bf4173919e3ed69467f408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:29:54.226325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:54.227008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:29:54.228612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1151/2279] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:29:54 2022] DELETE /identity/v3/projects/99ed0de2b2bf4173919e3ed69467f408 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:29:55.410519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:29:55.412117 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:55.503030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55c084-3a2e-4f4e-a840-3a5fe8eb1af0 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:29:55.575875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:29:55.576495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:29:55.576769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:29:55.577038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:29:55.579951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:29:55.645701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:29:55.647605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:29:55.657470 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1129/2280] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:29:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:04.385937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:05.652904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:12.665050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:12.667715 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:12.736429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abd09f-4ccb-46e4-9fe1-4056d4c41600 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:12.740120 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:12.741405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:12.741405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:12.741405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:12.743153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:12.761610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:12.763128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96068a06-d1ab-45e5-a6ac-323a6722378b service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:12.772726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1152/2281] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:17.839567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:17.841102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:17.856425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf8b61-49ce-495e-842a-d2a5b4c709a1 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:17.862970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:17.863595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:17.863875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:17.864145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:17.866014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:17.930138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:17.931762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:17.941562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1130/2282] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:22.768334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=26 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:23.539977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:23.540439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:23.540895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:23.563638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] MFA Rules not processed for user `7d2ee46e8302468bb90ee913eeeba4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:30:23.564884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:23.636996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1153/2283] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:30:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:30:23.986680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:23.988338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:24.001279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e1806-d2c8-4aa3-829c-5ed9e389dff7 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:24.005189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:24.006023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:24.006403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:24.006703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:24.008846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:24.020659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:24.022181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:24.033109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1131/2284] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:24.514585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:24.517467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:24.634645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a871ea-9aef-4257-8189-2cb7e2c64006 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:24.641406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:24.642374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:24.642649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:24.642919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:24.645386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:24.661080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:24.662897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471005d-2c07-4c69-bad3-f9d65af0906e service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:24.674609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1154/2285] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:26.074094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:26.088869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be2b60-71b7-4ae8-9a23-df19e5848cfc service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:26.095665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:26.097058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:26.097058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:26.097058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:26.099550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:26.113332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:26.115039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:26.130856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1132/2286] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:29.928525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:29.929092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:29.929092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:29.949168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] MFA Rules not processed for user `dc0bc75b6b3f4d1689bc1f7faaa2e5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:30:29.950406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:29.958006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1155/2287] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:30:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:30:36.125048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=40 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:37.195695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:37.215247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b65db-2eb0-40bc-920a-8ed3fb093eb1 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:37.234991 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:37.236148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:37.236650 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:37.237103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:37.240595 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:37.261591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:37.263851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:37.295435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1133/2288] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:39.943883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=60 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:45.967316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:30:45.967316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:46.053063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cff87c-09f3-4d3a-86e7-209f572b1111 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:30:46.064463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:30:46.065346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:30:46.065885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:30:46.066302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:30:46.070370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:30:46.088086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:30:46.096247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:30:46.130483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1156/2289] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:30:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:30:47.285883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:30:56.107850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=28 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:09.828640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:09.830539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:09.994039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dba25-28a2-4f7b-8f85-30efd0463126 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:09.998956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:09.999631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:09.999999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:10.000274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] PATH_INFO: `/v3/users/50dab514c64d47d095bd6d1ac8ce3b56` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:10.001495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50dab514c64d47d095bd6d1ac8ce3b56)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:10.003511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:10.247954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50dab514c64d47d095bd6d1ac8ce3b56'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.248538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50dab514c64d47d095bd6d1ac8ce3b56'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.324090 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50dab514c64d47d095bd6d1ac8ce3b56'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.333236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50dab514c64d47d095bd6d1ac8ce3b56'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.348368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-3e9cb310-6d25-4770-90a0-60b644141db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50dab514c64d47d095bd6d1ac8ce3b56'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.356911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1134/2290] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:31:09 2022] DELETE /identity/v3/users/50dab514c64d47d095bd6d1ac8ce3b56 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:31:10.733412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:10.733412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8444-2ee0-4da2-b92a-d6640681700a service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:10.758620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:10.759256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:10.759533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:10.759805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] PATH_INFO: `/v3/projects/cbffdee4dc4341e4a33fcfe884f28072` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:10.760942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbffdee4dc4341e4a33fcfe884f28072)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:10.762879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:10.873883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbffdee4dc4341e4a33fcfe884f28072', 'request_id': 'req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.881095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbffdee4dc4341e4a33fcfe884f28072', 'request_id': 'req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.956398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbffdee4dc4341e4a33fcfe884f28072', 'request_id': 'req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.967709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] The token cache is being invalidate because project cbffdee4dc4341e4a33fcfe884f28072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:31:10.967995 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.969214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b8a798ee-d45b-4d9c-b284-69043ca5ff7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:31:10.970839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1157/2291] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:31:10 2022] DELETE /identity/v3/projects/cbffdee4dc4341e4a33fcfe884f28072 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:31:10.977098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a760-97ee-40cb-9858-34f720702376 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:10.977891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a760-97ee-40cb-9858-34f720702376 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:10.977891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54a760-97ee-40cb-9858-34f720702376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.001551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:11.038379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-af54a760-97ee-40cb-9858-34f720702376 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:11.039806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-af54a760-97ee-40cb-9858-34f720702376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:11.099873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1135/2292] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:11.107820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.108098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.108365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.129263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:11.130412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:11.143363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1158/2293] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:11.147668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54a760-97ee-40cb-9858-34f720702376 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.158371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54a760-97ee-40cb-9858-34f720702376 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.167328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.167945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.168227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.168500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.169157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:31:11.169463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.171132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.184293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1136/2294] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:31:11.189344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.198738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a77fcb-43e1-4cb3-bc5f-6609d8f68a64 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.203978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.204611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.204888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.205162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.205821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.207722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.317312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1159/2295] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:31:11 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:31:11.321482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.332044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ded49-1491-4477-bcbf-5c3069fe10a0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.394893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.395423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.395705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.395982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.396630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.398624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.501925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1137/2296] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:31:11 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:31:11.507098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.518007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b63eb-7e6e-4820-882d-9bc8e31b043c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.523075 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.523695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.523974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.524251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.524902 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.526551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.538242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1160/2297] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:31:11.542721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.553795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e26c2-1b4b-481a-a3a6-df8b8c7f5177 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.558523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.559119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.559399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.559672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.560318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.562005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12dfa0-c5d2-4c69-a447-4dc78985ab83 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.573592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1138/2298] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:31:11.578036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.588340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233610e-9b71-4100-8e85-c36c845c4c67 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.594984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.595652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.595934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.596220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] PATH_INFO: `/v3/projects/beb5b0e674994eabb1e18fde391c2484/users/e9450f3f7527406ab70fcf0b64cab787/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.613037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb5b0e674994eabb1e18fde391c2484, user_id=e9450f3f7527406ab70fcf0b64cab787, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.615799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.707444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1161/2299] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:31:11 2022] PUT /identity/v3/projects/beb5b0e674994eabb1e18fde391c2484/users/e9450f3f7527406ab70fcf0b64cab787/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:31:11.713681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d319af-ba99-428a-9804-7b8d3260ffc2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.714008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d319af-ba99-428a-9804-7b8d3260ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.714291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d319af-ba99-428a-9804-7b8d3260ffc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.738370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-29d319af-ba99-428a-9804-7b8d3260ffc2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:11.739638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-29d319af-ba99-428a-9804-7b8d3260ffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:11.790486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:11.805176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1139/2300] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:11.815510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:11.817221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:11.839289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:11.897077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651e61c-93c2-466a-a898-18ac4f03cca4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:11.900686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:11.901284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:11.901556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:11.901857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:11.911788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:11.913220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:11.924079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1162/2301] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:31:18.004092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.016381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.016381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.056508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] MFA Rules not processed for user `e9450f3f7527406ab70fcf0b64cab787`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:18.058025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:18.150110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1140/2302] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:18.154962 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.165793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a256b-d991-451a-91f8-aa8c83a76c98 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.171209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.172622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.172994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.173524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.174370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.176532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.365698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1163/2303] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:31:18 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:31:18.371830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.383630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebd8ac-2c6c-4786-a912-7fcb453012ba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.456254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.456886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.457164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.457439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.458129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.460123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.581556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1141/2304] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:31:18 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:31:18.585698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.596741 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007653ef-5d3f-4070-ab6c-daf5387a9b1d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.601483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.602158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.602514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.602795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.603445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.605217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.617432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1164/2305] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:31:18.622474 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.633141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60e0cd-d3d0-4f38-81ad-4a8ed2f7d644 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.638361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.639026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.639302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.639573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] PATH_INFO: `/v3/projects/234c29fb02204981b2f2fb131b95282f/users/2f6b9c0532904f899ab6266e7b65a7f5/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.659380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=234c29fb02204981b2f2fb131b95282f, user_id=2f6b9c0532904f899ab6266e7b65a7f5, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.662147 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.754324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1142/2306] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:31:18 2022] PUT /identity/v3/projects/234c29fb02204981b2f2fb131b95282f/users/2f6b9c0532904f899ab6266e7b65a7f5/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:31:18.759614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.771793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cd384-9611-4a43-afa1-b5b639930f20 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.838284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.838898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.839207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.839419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.840280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.841678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9a4f8-1949-4a90-994f-b7b0403f9c2f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.854539 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1165/2307] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:31:18.858731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:18.880177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fce5c1-ce02-4e3d-bdbd-62d8dd009c8f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:18.885196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:18.886002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:18.886211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:18.886480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] PATH_INFO: `/v3/projects/234c29fb02204981b2f2fb131b95282f/users/2f6b9c0532904f899ab6266e7b65a7f5/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:18.888551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=234c29fb02204981b2f2fb131b95282f, user_id=2f6b9c0532904f899ab6266e7b65a7f5, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:18.892393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:18.907237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1143/2308] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:31:18 2022] PUT /identity/v3/projects/234c29fb02204981b2f2fb131b95282f/users/2f6b9c0532904f899ab6266e7b65a7f5/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:31:26.868347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933c86-212e-4daf-b8d9-a720abeeaa36 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:26.868907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933c86-212e-4daf-b8d9-a720abeeaa36 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:26.868907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d933c86-212e-4daf-b8d9-a720abeeaa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:26.930699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-1d933c86-212e-4daf-b8d9-a720abeeaa36 None None] MFA Rules not processed for user `2f6b9c0532904f899ab6266e7b65a7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:26.932808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1d933c86-212e-4daf-b8d9-a720abeeaa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:27.015340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:27.042445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1166/2309] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:27.057209 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:27.059512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:27.108794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:27.146740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ac1a6-346d-4ec6-b450-5a901c2efedb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:27.157621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:27.157621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:27.157621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:27.157621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:27.212902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:27.214589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:27.226097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1144/2310] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:31:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:31:30.074439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613f5dd-7237-4205-9367-bb5512338578 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:30.075755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613f5dd-7237-4205-9367-bb5512338578 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:30.076224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613f5dd-7237-4205-9367-bb5512338578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:30.110864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d613f5dd-7237-4205-9367-bb5512338578 None None] MFA Rules not processed for user `e4f87633619b42d989942e1d63d30fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:30.112934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d613f5dd-7237-4205-9367-bb5512338578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:30.194317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1167/2311] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:31:30.603039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:31:30.672367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95e60d-3818-4f0c-ae43-be93ce833912 service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:31:30.676196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:31:30.676808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:30.677087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:30.677354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:30.679148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:30.690604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:31:30.692114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d737a-7147-48b1-9823-355681181c30 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:31:30.701809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1145/2312] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:31:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:31:40.185288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=30 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:40.697428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=66 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:31:57.678573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:31:57.679212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:31:57.679212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:31:57.701267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] MFA Rules not processed for user `e9450f3f7527406ab70fcf0b64cab787`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:31:57.702554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:31:57.710482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1168/2313] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:31:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:32:07.694008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:11.262150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:11.262732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:11.262732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:11.282789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] MFA Rules not processed for user `2ca47c37cd2d4999afa598cac6daa2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:11.283893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:11.341108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1146/2314] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:19.644652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:19.646380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:19.660862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aebec-9e37-455d-8615-84ff39d57181 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:19.664865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:19.665490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:19.665804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:19.666074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:19.669351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:19.752517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:19.754227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:19.766572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1169/2315] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:21.333791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=25 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:23.942063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:23.943584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:24.050840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ae3cb-273d-49a4-80d6-dc47e1cca929 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:24.056242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:24.056878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:24.057239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:24.057518 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:24.059404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:24.071684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:24.073280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:24.087233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1147/2316] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:24.094470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:24.097512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:24.122069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634cfa9-67f9-4869-bc8f-8cfec7c3421a service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:24.132535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:24.133339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:24.133658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:24.133998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] PATH_INFO: `/v3/users/dc0bc75b6b3f4d1689bc1f7faaa2e5bc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:24.135279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0bc75b6b3f4d1689bc1f7faaa2e5bc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:24.138209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:24.369317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0bc75b6b3f4d1689bc1f7faaa2e5bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:24.370560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0bc75b6b3f4d1689bc1f7faaa2e5bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:24.384247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0bc75b6b3f4d1689bc1f7faaa2e5bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:24.513527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0bc75b6b3f4d1689bc1f7faaa2e5bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:24.523536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0bc75b6b3f4d1689bc1f7faaa2e5bc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:24.545069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1170/2317] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:32:24 2022] DELETE /identity/v3/users/dc0bc75b6b3f4d1689bc1f7faaa2e5bc => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:24.630791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:24.640538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e27-e047-4b30-9c7a-4fa2f7280c98 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:24.701876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:24.701876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:24.701876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:24.702308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:24.704306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:24.795216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:24.796833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4519084-c998-410d-91d7-8d798db3d60d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:24.810565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1148/2318] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:24.848878 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:24.850947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:24.950131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344f6d-34a0-4d69-acf1-4514f9b7bd14 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:24.955684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:24.956537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:24.956988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:24.957266 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] PATH_INFO: `/v3/projects/786534cce098405196143139d375b465` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:24.958421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786534cce098405196143139d375b465)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:24.960412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:24.968926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:25.105532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786534cce098405196143139d375b465', 'request_id': 'req-84f13062-426c-41ee-9a25-5726c68f8601'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:25.112500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786534cce098405196143139d375b465', 'request_id': 'req-84f13062-426c-41ee-9a25-5726c68f8601'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:25.243645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786534cce098405196143139d375b465', 'request_id': 'req-84f13062-426c-41ee-9a25-5726c68f8601'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:25.256585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] The token cache is being invalidate because project 786534cce098405196143139d375b465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:32:25.257139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:25.258150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-84f13062-426c-41ee-9a25-5726c68f8601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:25.260678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1171/2319] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:32:24 2022] DELETE /identity/v3/projects/786534cce098405196143139d375b465 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:25.266303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.266808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.268287 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.296618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:25.297854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:25.315239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:25.366016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1149/2320] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:32:25.373636 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.373955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.374232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.394809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:25.396039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:25.403826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1172/2321] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:25.408409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:25.419339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4d85d-7944-4346-98f9-c66fcec2cd34 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:25.424128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:25.424730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.436172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.436172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.436172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:32:25.436172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:25.436172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:25.440528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1150/2322] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:32:25.445213 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:25.454472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566319b-7cda-4b11-850e-6a255f69ed92 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:25.459839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:25.460432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.460708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.460979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.461601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:25.463527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:25.709874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1173/2323] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:32:25 2022] POST /identity/v3/projects => generated 363 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:32:25.714116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:25.725451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e61df0-276c-487e-8983-acb44c12b0ae admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:25.794195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:25.794928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.795206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.795515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.796222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:25.798430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:25.946716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1151/2324] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:32:25 2022] POST /identity/v3/users => generated 282 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:32:25.948676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:25.960089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae30f-66c2-485d-b66f-c7e90c15ae4d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:25.964773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:25.965394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:25.968486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:25.968486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:25.968486 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:25.968970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:25.983994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:25.991152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1174/2325] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:25.994858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d74a7-efbb-442b-9ad8-772b768cbffd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:26.000131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:26.000763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:26.001193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:26.001363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:26.002080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:26.006645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7e14f-6e0d-4a8a-83ed-8cd473ce533c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:26.022956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1152/2326] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:26.027448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:26.041008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4460550-9e69-492d-b9af-93c7d9e63269 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:26.047349 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:26.048133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:26.048497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:26.049211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] PATH_INFO: `/v3/projects/0f1d9fa32da24e9999721eb3cc95aa0a/users/a36d3c539520468b963a8ee082534ad7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:26.078967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1d9fa32da24e9999721eb3cc95aa0a, user_id=a36d3c539520468b963a8ee082534ad7, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:26.084806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:26.239504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1175/2327] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:32:26 2022] PUT /identity/v3/projects/0f1d9fa32da24e9999721eb3cc95aa0a/users/a36d3c539520468b963a8ee082534ad7/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:32:26.250901 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:26.251222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:26.251500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:26.283224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] MFA Rules not processed for user `a36d3c539520468b963a8ee082534ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:26.285014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:26.357624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1153/2328] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:32:26.368430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:26.370204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:26.454608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9980f01-42b8-4b39-b45d-fa1d56330120 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:26.458088 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:26.458683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:26.458956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:26.459221 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:26.467775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:26.469188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:26.478979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1176/2329] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:33.295573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:33.297049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:33.317608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:33.404555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c54f0-579a-4413-b605-3d31ca2ac891 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:33.411798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:33.412417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:33.412810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:33.413124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:33.415069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:33.478987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:33.480583 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:33.490508 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1154/2330] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:36.474960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:37.725685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:37.730524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:37.811499 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3ccfa-0253-420e-a459-4464b8b826a0 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:37.817757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:37.818378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:37.818671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:37.818921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:37.820879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:37.833879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:37.835479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd1131-3e1a-4ce1-946f-4314c57acf0c service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:37.846164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1177/2331] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:40.186588 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:40.251618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91073d-2897-47de-93d1-cea1863d702b service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:40.256749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:40.257434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:40.257780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:40.258342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] PATH_INFO: `/v3/users/a36d3c539520468b963a8ee082534ad7` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:40.259610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36d3c539520468b963a8ee082534ad7)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:40.261600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:40.444950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:40.637994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36d3c539520468b963a8ee082534ad7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:40.639019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36d3c539520468b963a8ee082534ad7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:40.818785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36d3c539520468b963a8ee082534ad7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:40.826950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36d3c539520468b963a8ee082534ad7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:40.839218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36d3c539520468b963a8ee082534ad7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:40.846426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1155/2332] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:32:40 2022] DELETE /identity/v3/users/a36d3c539520468b963a8ee082534ad7 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:40.853037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:40.853554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:40.853554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:40.873644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:40.874892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:40.938538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1178/2333] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:40.948576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:40.950030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:41.013215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac24627-c014-49b5-9052-93d8c98e1be2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:41.016835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:41.017435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:41.017728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:41.018018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:41.027132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:41.028553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb65b0-1d08-4abe-a1f7-099510dbb07e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:41.039388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1156/2334] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:41.560173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:41.561685 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:41.574604 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdf615-105c-472b-bd23-91018ab77af1 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:41.578774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:41.579409 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:41.579677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:41.579947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] PATH_INFO: `/v3/projects/0f1d9fa32da24e9999721eb3cc95aa0a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:41.581067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1d9fa32da24e9999721eb3cc95aa0a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:41.583906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:41.752937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1d9fa32da24e9999721eb3cc95aa0a', 'request_id': 'req-9ef91d32-1074-472a-a3db-a9a953a0f972'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:41.759692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1d9fa32da24e9999721eb3cc95aa0a', 'request_id': 'req-9ef91d32-1074-472a-a3db-a9a953a0f972'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:41.846734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1d9fa32da24e9999721eb3cc95aa0a', 'request_id': 'req-9ef91d32-1074-472a-a3db-a9a953a0f972'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:41.858421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] The token cache is being invalidate because project 0f1d9fa32da24e9999721eb3cc95aa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:32:41.860584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:41.860584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9ef91d32-1074-472a-a3db-a9a953a0f972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:32:41.860844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1179/2335] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:32:41 2022] DELETE /identity/v3/projects/0f1d9fa32da24e9999721eb3cc95aa0a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:41.866778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:41.867298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:41.867420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:41.887495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:41.888697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:41.950942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1157/2336] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:41.959711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:41.960036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:41.960302 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:41.981540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:41.982782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:41.991365 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1180/2337] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:41.995738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.005596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bc0df-1df1-405c-aa7a-dab32b885962 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.010469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.011087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.011368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.011700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.012476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:32:42.012783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.014495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.029994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1158/2338] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:32:42.035356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.044904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc933d-2598-499f-9c73-d5a97b4dd7b6 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.049696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.050412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.050673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.050948 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.051582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.053471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.058070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:42.185832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1181/2339] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:32:42 2022] POST /identity/v3/projects => generated 357 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:32:42.190195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.200160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87626-4604-4e3a-b14f-7e66a17229d4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.234260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:42.261451 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.262123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.262410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.262663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.263305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.265284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.356107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1159/2340] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:32:42 2022] POST /identity/v3/users => generated 279 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:32:42.360556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.371034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c314887-4946-4552-8605-c341222752f1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.375757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.376356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.376638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.376901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.377537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.379887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.390184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1182/2341] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:42.395423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.405983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3df985-c646-4d50-a864-9b5946890bb5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.411539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.412450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.412748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.413019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.413671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.415356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bfa67-46b3-473b-9f55-375cfaacb136 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.426448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1160/2342] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:42.430345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.441401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006064ec-12cf-4226-90a5-ea2d5208fab8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.446023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.446681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.446961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.447223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] PATH_INFO: `/v3/projects/77903817d817490eaff355eb9c5b095c/users/d8b8800acd5441118c0612e561098b7e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.465086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=77903817d817490eaff355eb9c5b095c, user_id=d8b8800acd5441118c0612e561098b7e, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.467821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.571710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1183/2343] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:32:42 2022] PUT /identity/v3/projects/77903817d817490eaff355eb9c5b095c/users/d8b8800acd5441118c0612e561098b7e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:42.582534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.582830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.583093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.612609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] MFA Rules not processed for user `d8b8800acd5441118c0612e561098b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:42.614434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:42.690471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1161/2344] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:42.696013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.773089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc8aa8-2b15-4174-bcf4-d73b8ceabc0e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.778070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.778642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.778958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.779233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.779860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.782039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:42.869463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1184/2345] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:32:42 2022] POST /identity/v3/projects => generated 357 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:32:42.873838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:42.885397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac94f5c-13c7-4fe5-ba06-d7982584e598 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:42.948585 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:42.949130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:42.949414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:42.949793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:42.950460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:42.952497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.084331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1162/2346] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:32:42 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:32:43.085183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.096058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813a5e-3748-48f1-b619-bfacea90add3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.101380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.111682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.111682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.111682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.111682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.111682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.117223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1185/2347] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:43.122431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.133964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df35d6-b03c-485c-9e20-ab35f955c73b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.138762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.139536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.139879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.140218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] PATH_INFO: `/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b/users/b12f35dc361044e4a7fc83d4203f856a/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.157013 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b62b0ea4fdb41758bcd0434696cf57b, user_id=b12f35dc361044e4a7fc83d4203f856a, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.160577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.288710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1163/2348] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:32:43 2022] PUT /identity/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b/users/b12f35dc361044e4a7fc83d4203f856a/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:43.292886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.304048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531b018-947a-4465-a2b4-dda85d4f7fe6 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.367389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.368052 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.368291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.368566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.369214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.370855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643e4e5-b61e-43a7-a6ef-0ab1bf06ca7b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.381923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1186/2349] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:32:43.386175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.398346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b09b4-88e9-4290-8b53-465674834788 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.403376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.404034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.404311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.404581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] PATH_INFO: `/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b/users/b12f35dc361044e4a7fc83d4203f856a/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.406814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b62b0ea4fdb41758bcd0434696cf57b, user_id=b12f35dc361044e4a7fc83d4203f856a, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.409484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.414046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:43.421862 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1164/2350] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:32:43 2022] PUT /identity/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b/users/b12f35dc361044e4a7fc83d4203f856a/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:32:43.434530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.434819 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.435096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.465065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] MFA Rules not processed for user `b12f35dc361044e4a7fc83d4203f856a`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:43.466341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:43.475887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:32:43.541507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1187/2351] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:43.554241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.556042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:43.626704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ad14d-3df1-4247-a370-802f81096686 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.631478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.632104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.632384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.632686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.642658 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.644180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a5800-a5e6-4315-aae5-df5b9db8d0ad service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:43.654930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1165/2352] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:43.921833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:32:43.932141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2e094-4432-406d-9832-b2db21eb8d25 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:32:43.935858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:32:43.936449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:43.936722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:43.937010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:43.996339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:32:43.997790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:32:44.007087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1188/2353] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:32:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:32:51.645460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901e7-0e5e-4ce6-a53f-ca6cb6a70925 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:32:51.646101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901e7-0e5e-4ce6-a53f-ca6cb6a70925 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:32:51.646101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0901e7-0e5e-4ce6-a53f-ca6cb6a70925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:32:51.667120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3c0901e7-0e5e-4ce6-a53f-ca6cb6a70925 None None] MFA Rules not processed for user `7d2ee46e8302468bb90ee913eeeba4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:32:51.669229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3c0901e7-0e5e-4ce6-a53f-ca6cb6a70925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:32:51.741534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1166/2354] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:32:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:32:54.002688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=53 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:01.730305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=53 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:07.549047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:07.619676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62034bd8-2d82-4002-a8be-f586f1b9703f service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:07.624363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:07.627825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:07.627825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:07.627825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] PATH_INFO: `/v3/users/d8b8800acd5441118c0612e561098b7e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:07.627825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b8800acd5441118c0612e561098b7e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:07.629862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:07.872796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b8800acd5441118c0612e561098b7e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:07.873428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b8800acd5441118c0612e561098b7e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:07.881192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b8800acd5441118c0612e561098b7e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:08.000692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b8800acd5441118c0612e561098b7e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:08.004428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b8800acd5441118c0612e561098b7e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:08.021593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1189/2355] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:07 2022] DELETE /identity/v3/users/d8b8800acd5441118c0612e561098b7e => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:33:08.026678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:08.027214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:08.027214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:08.048016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:08.049190 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:08.122998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1167/2356] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:33:08.142640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:08.144239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:08.209882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935d243-db3e-4ab8-a12f-8d04c6a4ac40 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:08.214135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:08.214760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:08.215038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:08.215300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:08.228547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:08.230210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:08.242689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1190/2357] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:08.762532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:08.764029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:08.779270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6474bfb-ebe3-4b94-a086-6fbf258ae47e None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:08.782890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:08.783587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:08.783833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:08.784100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] PATH_INFO: `/v3/projects/77903817d817490eaff355eb9c5b095c` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:08.785990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77903817d817490eaff355eb9c5b095c)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:08.787972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:08.932198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77903817d817490eaff355eb9c5b095c', 'request_id': 'req-9e0fff3a-e564-49fe-96a8-017aed153f10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:08.939786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77903817d817490eaff355eb9c5b095c', 'request_id': 'req-9e0fff3a-e564-49fe-96a8-017aed153f10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.038220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77903817d817490eaff355eb9c5b095c', 'request_id': 'req-9e0fff3a-e564-49fe-96a8-017aed153f10'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.049927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] The token cache is being invalidate because project 77903817d817490eaff355eb9c5b095c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:09.050240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.051072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.052594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1168/2358] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:08 2022] DELETE /identity/v3/projects/77903817d817490eaff355eb9c5b095c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:09.056649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:09.067652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fece8-0052-4269-9e78-f941dc78527e service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:09.089482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:09.132430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:09.133072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:09.133348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:09.133617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] PATH_INFO: `/v3/users/b12f35dc361044e4a7fc83d4203f856a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:09.134827 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12f35dc361044e4a7fc83d4203f856a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:09.137036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:09.391465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12f35dc361044e4a7fc83d4203f856a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.392128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12f35dc361044e4a7fc83d4203f856a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.400686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12f35dc361044e4a7fc83d4203f856a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.495447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12f35dc361044e4a7fc83d4203f856a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.496079 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-999e6530-7baf-4ae5-b508-ff6162430f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12f35dc361044e4a7fc83d4203f856a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:09.516480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1191/2359] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:09 2022] DELETE /identity/v3/users/b12f35dc361044e4a7fc83d4203f856a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:09.967341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:09.969830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:09.992227 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fff3a-e564-49fe-96a8-017aed153f10 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.055863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.056559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.056784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.057053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] PATH_INFO: `/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.058333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b62b0ea4fdb41758bcd0434696cf57b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.060204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.203498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:10.220454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b62b0ea4fdb41758bcd0434696cf57b', 'request_id': 'req-b387cfc6-bf15-48d3-9faa-ee29df000d8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:10.227312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b62b0ea4fdb41758bcd0434696cf57b', 'request_id': 'req-b387cfc6-bf15-48d3-9faa-ee29df000d8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:10.302352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b62b0ea4fdb41758bcd0434696cf57b', 'request_id': 'req-b387cfc6-bf15-48d3-9faa-ee29df000d8f'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:10.314502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] The token cache is being invalidate because project 0b62b0ea4fdb41758bcd0434696cf57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:10.314934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:10.315785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b387cfc6-bf15-48d3-9faa-ee29df000d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:10.317504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1169/2360] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:09 2022] DELETE /identity/v3/projects/0b62b0ea4fdb41758bcd0434696cf57b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:10.323047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.323554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.324745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.345484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:10.347262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:10.417789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1192/2361] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:33:10.418438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.422137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.422137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.445198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:10.445198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:10.451651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1170/2362] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:33:10.457126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.468759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eb4a4-faf0-4d16-b114-8eb13efe8575 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.474993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.475773 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.476225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.476643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.477447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:33:10.477936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.479616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.493547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1193/2363] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:33:10.498723 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.508102 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741e09e-2c90-4bc1-9b4c-c0111c59d47c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.512735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.513333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.513605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.513897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.514528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.516418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.625126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1171/2364] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:10 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:10.631374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.639982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af1f24-9e95-46fd-8633-ac09ea517bb6 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.672274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:10.713694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.714437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.715377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.715717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.716419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.718477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.852657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1194/2365] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:10 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:10.856937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.869908 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484e669-aca0-43b5-9da3-d3c46db17861 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.876075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.876676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.876946 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.877220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.877911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.880586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.891926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1172/2366] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:10.899122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.912355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c80d3-508d-40ca-8c3f-ed1604c31432 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.917270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.918069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.918347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.918618 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:10.920240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:10.921134 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b147a-dc59-4b4a-a9e0-4db282b6fdfa admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:10.932348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1195/2367] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:10.936731 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:10.956818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2e014-7b3c-4c20-8ed7-74cec38ffe01 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:10.965975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:10.966997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:10.967479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:10.967962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] PATH_INFO: `/v3/projects/e977a04bccea4fa5a78b2c7fdae5d1a9/users/0e6a9163efff491eb4a1e0af117fa283/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.000950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e977a04bccea4fa5a78b2c7fdae5d1a9, user_id=0e6a9163efff491eb4a1e0af117fa283, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:11.007414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:11.168524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1173/2368] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:10 2022] PUT /identity/v3/projects/e977a04bccea4fa5a78b2c7fdae5d1a9/users/0e6a9163efff491eb4a1e0af117fa283/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:11.180696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:11.181062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:11.181342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.212278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] MFA Rules not processed for user `0e6a9163efff491eb4a1e0af117fa283`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:11.213527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:11.285474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1196/2369] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:11.293065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:11.375208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddff7d-4682-4c87-8542-6a39361e1231 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:11.381037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:11.381652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:11.381964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:11.382487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.382929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:11.384857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:11.533328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1174/2370] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:11 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:33:11.537255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:11.548087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f61ea-07d0-45db-bc3a-f40000f6a90b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:11.619619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:11.620241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:11.620515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:11.620780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.621403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:11.623366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:11.764702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1197/2371] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:11 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:11.769811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:11.785357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b151651-f183-4a30-86e3-3a4c3e813c4f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:11.790573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:11.791233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:11.791507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:11.791850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.792506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:11.794278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:11.811368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1175/2372] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:33:11.811947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:11.823152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754cdfe-74aa-4e63-a398-d0cc6376168a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:11.827875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:11.828570 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:11.828845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:11.829111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] PATH_INFO: `/v3/projects/d2f9731e6545415e9358504d8db12aa9/users/f8ca692437524067b7893713ea360bee/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:11.844449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f9731e6545415e9358504d8db12aa9, user_id=f8ca692437524067b7893713ea360bee, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:11.847665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:11.933157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1198/2373] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:11 2022] PUT /identity/v3/projects/d2f9731e6545415e9358504d8db12aa9/users/f8ca692437524067b7893713ea360bee/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:11.937219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:11.947438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725649e-357d-4f9f-829a-ca162403db1c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:11.968216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:12.005687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:12.006377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.006662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.006942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.007599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:12.009232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11859a44-ebfd-4275-8c4b-24e5a25cff35 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:12.022096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1176/2374] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:12.026293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:12.032343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:12.038552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1d5fb-8bca-445c-b9d0-913cc02f2818 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:12.043757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:12.044453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.044808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.045083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] PATH_INFO: `/v3/projects/d2f9731e6545415e9358504d8db12aa9/users/f8ca692437524067b7893713ea360bee/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.047102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f9731e6545415e9358504d8db12aa9, user_id=f8ca692437524067b7893713ea360bee, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:12.050856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:12.060631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1199/2375] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:12 2022] PUT /identity/v3/projects/d2f9731e6545415e9358504d8db12aa9/users/f8ca692437524067b7893713ea360bee/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:12.071142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.071428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.071705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.100040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] MFA Rules not processed for user `f8ca692437524067b7893713ea360bee`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:12.101260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:12.172300 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1177/2376] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:12.183982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:12.185689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:12.246556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05631514-8337-4615-b6bd-2dfac5645d84 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:12.250345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:12.250953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.251233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.251506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.261908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:12.263359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdff4-02f2-465a-95ac-2781e0fcd331 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:12.274834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1200/2377] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:12.346402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:12.410060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6627cb1-ac17-4ad1-8bc0-aa16166a63a5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:12.414770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:12.415398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.415679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.415953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] PATH_INFO: `/v3/users/0e6a9163efff491eb4a1e0af117fa283` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.417107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e6a9163efff491eb4a1e0af117fa283)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:12.419063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:12.639491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e6a9163efff491eb4a1e0af117fa283'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:12.640155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e6a9163efff491eb4a1e0af117fa283'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:12.641867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:12.760657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e6a9163efff491eb4a1e0af117fa283'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:12.768565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e6a9163efff491eb4a1e0af117fa283'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:12.780685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e6a9163efff491eb4a1e0af117fa283'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:12.787270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1178/2378] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:12 2022] DELETE /identity/v3/users/0e6a9163efff491eb4a1e0af117fa283 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:12.792287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.792778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.792884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.812305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:12.813571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:12.873738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1201/2379] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:12.885358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:12.886803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:12.944066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f25a-a043-4e01-b352-593fd8067213 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:12.947738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:12.948343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:12.948617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:12.948888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:12.961748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:12.963245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:12.984655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1179/2380] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:13.486734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:13.488996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:13.504836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871194f8-b7d6-4b5e-88c8-80b780a2e1db None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:13.509939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:13.510702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:13.511042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:13.511380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] PATH_INFO: `/v3/projects/e977a04bccea4fa5a78b2c7fdae5d1a9` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:13.512725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e977a04bccea4fa5a78b2c7fdae5d1a9)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:13.514731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:13.624494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e977a04bccea4fa5a78b2c7fdae5d1a9', 'request_id': 'req-5f38fa85-cbc7-4d13-865d-fb48173afa74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:13.631227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e977a04bccea4fa5a78b2c7fdae5d1a9', 'request_id': 'req-5f38fa85-cbc7-4d13-865d-fb48173afa74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:13.709275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e977a04bccea4fa5a78b2c7fdae5d1a9', 'request_id': 'req-5f38fa85-cbc7-4d13-865d-fb48173afa74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:13.720969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] The token cache is being invalidate because project e977a04bccea4fa5a78b2c7fdae5d1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:13.721258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:13.722507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:13.724020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1202/2381] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:13 2022] DELETE /identity/v3/projects/e977a04bccea4fa5a78b2c7fdae5d1a9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:13.728533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:13.738874 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518802f-33cb-4ad1-81b0-091727115dc1 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:13.798559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:13.799088 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:13.799364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:13.799636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] PATH_INFO: `/v3/users/f8ca692437524067b7893713ea360bee` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:13.801449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ca692437524067b7893713ea360bee)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:13.803417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:14.004409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ca692437524067b7893713ea360bee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.005038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ca692437524067b7893713ea360bee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.069957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ca692437524067b7893713ea360bee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.080453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ca692437524067b7893713ea360bee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.087781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:14.094582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7ea08b8e-adda-4dad-b836-0f8d0fc64bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ca692437524067b7893713ea360bee'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.100969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1180/2382] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:13 2022] DELETE /identity/v3/users/f8ca692437524067b7893713ea360bee => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:14.591455 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:14.593351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:14.611981 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38fa85-cbc7-4d13-865d-fb48173afa74 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:14.627530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:14.674675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:14.677246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:14.677246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:14.677246 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] PATH_INFO: `/v3/projects/d2f9731e6545415e9358504d8db12aa9` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:14.677740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f9731e6545415e9358504d8db12aa9)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:14.679987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:14.826891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f9731e6545415e9358504d8db12aa9', 'request_id': 'req-280fec23-29f8-4183-b6b3-42cde8ec00f6'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.833494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f9731e6545415e9358504d8db12aa9', 'request_id': 'req-280fec23-29f8-4183-b6b3-42cde8ec00f6'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.937961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f9731e6545415e9358504d8db12aa9', 'request_id': 'req-280fec23-29f8-4183-b6b3-42cde8ec00f6'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.949399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] The token cache is being invalidate because project d2f9731e6545415e9358504d8db12aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:14.949687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.950351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-280fec23-29f8-4183-b6b3-42cde8ec00f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:14.952114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1203/2383] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:14 2022] DELETE /identity/v3/projects/d2f9731e6545415e9358504d8db12aa9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:14.957782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:14.958286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:14.958286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:14.980729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:14.982016 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:15.042413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1181/2384] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:15.050444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.050620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.050857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.070559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:15.071776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:15.079717 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1204/2385] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:15.084482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.094059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68963195-7542-46f6-a1d8-360c624a84d5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.099565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.100155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.100431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.100700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.101339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:33:15.101633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.103280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.115532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1182/2386] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:33:15.119848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.129045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e44dc-16ce-4d3f-8d98-ebd08e230e0c None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.133616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.134329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.134586 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.134849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.135464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.137330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.263041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1205/2387] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:15 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:33:15.267274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.278092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660977b-7315-4deb-8057-8c8dc766d1c6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.339306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.339934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.340213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.340484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.341126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.343206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.464686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1183/2388] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:15 2022] POST /identity/v3/users => generated 303 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:15.469051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.480202 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08886f51-eced-4914-b1bc-93ab5e1697dd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.484823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.485434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.485795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.486018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.486663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.488276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.498804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1206/2389] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:15.504360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.514810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bf0f4-9966-4e02-9aeb-9fc431998fa9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.520151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.520758 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.521250 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.521528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.522290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.523969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d12ddf-6598-46cd-b068-bd124edf1108 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.535833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1184/2390] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:33:15.540069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.550597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33817c-d005-4344-a2f3-45d565f142f7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.555261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.555926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.556198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.556465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] PATH_INFO: `/v3/projects/3d0e10b4f3274ecbba8d23e9a0e4e3a6/users/5665f8bb48794b3596af3c29ab99e274/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.574606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0e10b4f3274ecbba8d23e9a0e4e3a6, user_id=5665f8bb48794b3596af3c29ab99e274, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.577354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:15.715033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1207/2391] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:15 2022] PUT /identity/v3/projects/3d0e10b4f3274ecbba8d23e9a0e4e3a6/users/5665f8bb48794b3596af3c29ab99e274/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:15.738609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.739138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.739930 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.786329 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] MFA Rules not processed for user `5665f8bb48794b3596af3c29ab99e274`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:15.788219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:15.847489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:15.873271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1185/2392] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:15.878128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:15.926059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:15.944222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b505a3-9411-4483-9ffa-f5259004a20c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:15.949193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:15.949842 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:15.950119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:15.950386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:15.951027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:15.953153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.119025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1208/2393] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:15 2022] POST /identity/v3/projects => generated 407 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:33:16.123809 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.136022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e5f70-a55b-43c6-ba32-1b651de1177c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.199480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.200487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.200980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.201337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.202106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.204124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.324710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1186/2394] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:16 2022] POST /identity/v3/users => generated 310 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:33:16.328879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.342300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d33f2-90b8-4277-a3f3-f8d780b05a2f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.347581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.348192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.348481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.348772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.349478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.351165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.361906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1209/2395] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:16.366453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.376897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbecc8-3912-4fc0-85a7-4c42eb84c9bc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.382191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.382840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.383116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.383384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] PATH_INFO: `/v3/projects/45a3a76953804bf1aaaafa65a1451b2d/users/83eb6c5bcd3741c395f2f9686518eaf7/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.399128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a3a76953804bf1aaaafa65a1451b2d, user_id=83eb6c5bcd3741c395f2f9686518eaf7, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.401798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.527633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1187/2396] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:16 2022] PUT /identity/v3/projects/45a3a76953804bf1aaaafa65a1451b2d/users/83eb6c5bcd3741c395f2f9686518eaf7/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:16.532043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.542975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065bba-4aa8-4734-97c1-31a6a40ab70b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.602614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.603310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.603593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.603870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.604514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.606181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037e634-0b75-494c-b225-82f431396096 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.620937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1210/2397] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:16.622264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.633403 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05ff84-c1ef-47e0-b819-715e86155dab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.637918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.638532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.638805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.639075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] PATH_INFO: `/v3/projects/45a3a76953804bf1aaaafa65a1451b2d/users/83eb6c5bcd3741c395f2f9686518eaf7/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.641330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a3a76953804bf1aaaafa65a1451b2d, user_id=83eb6c5bcd3741c395f2f9686518eaf7, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.644197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.654654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1188/2398] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:16 2022] PUT /identity/v3/projects/45a3a76953804bf1aaaafa65a1451b2d/users/83eb6c5bcd3741c395f2f9686518eaf7/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:16.665091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.665375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.665648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.694411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] MFA Rules not processed for user `83eb6c5bcd3741c395f2f9686518eaf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:16.695611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:16.764451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1211/2399] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:16.775938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:16.777664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:16.835952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c441-f377-40a1-8c0f-57fcdb1dc774 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:16.839616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:16.840205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:16.840478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:16.840745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:16.850223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:16.851633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:16.860742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1189/2400] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:16 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:17.266196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:17.277360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e08e0-039e-4f08-9159-c615b7de6a93 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:17.280969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:17.281898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:17.282174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:17.282441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:17.320344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:17.345952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:17.347965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f8398-afc9-435a-87bd-7a9f7fde56f9 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:17.357092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1212/2401] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:26.856663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:27.613705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=11 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:29.875076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:29.943576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969084df-0871-4f23-8b92-641e0e6c3b11 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:29.948283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:29.948923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:29.949201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:29.949500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] PATH_INFO: `/v3/users/5665f8bb48794b3596af3c29ab99e274` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:29.950671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5665f8bb48794b3596af3c29ab99e274)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:29.952662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:30.159596 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5665f8bb48794b3596af3c29ab99e274'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:30.160824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5665f8bb48794b3596af3c29ab99e274'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:30.229254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5665f8bb48794b3596af3c29ab99e274'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:30.236900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5665f8bb48794b3596af3c29ab99e274'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:30.249221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5665f8bb48794b3596af3c29ab99e274'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:30.255401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1190/2402] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:29 2022] DELETE /identity/v3/users/5665f8bb48794b3596af3c29ab99e274 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:30.259986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5467f-1485-429b-ba79-71143927b542 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:30.260493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5467f-1485-429b-ba79-71143927b542 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:30.260601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5467f-1485-429b-ba79-71143927b542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:30.280889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-31b5467f-1485-429b-ba79-71143927b542 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:30.282425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-31b5467f-1485-429b-ba79-71143927b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:30.341572 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1213/2403] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:30.351449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:30.352855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:30.411648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f61ee-ad07-45f9-b857-657294ce2aa1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:30.415313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:30.415923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:30.416200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:30.416472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:30.427436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:30.429338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:30.439113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1191/2404] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:30.876884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5467f-1485-429b-ba79-71143927b542 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:30.878359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-31b5467f-1485-429b-ba79-71143927b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:30.899889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5467f-1485-429b-ba79-71143927b542 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:30.907229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:30.907986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:30.908250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:30.908519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] PATH_INFO: `/v3/projects/3d0e10b4f3274ecbba8d23e9a0e4e3a6` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:30.909660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0e10b4f3274ecbba8d23e9a0e4e3a6)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:30.911591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:31.030211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0e10b4f3274ecbba8d23e9a0e4e3a6', 'request_id': 'req-693bfae6-5beb-4327-8572-92ce9a428841'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.037436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0e10b4f3274ecbba8d23e9a0e4e3a6', 'request_id': 'req-693bfae6-5beb-4327-8572-92ce9a428841'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.129488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0e10b4f3274ecbba8d23e9a0e4e3a6', 'request_id': 'req-693bfae6-5beb-4327-8572-92ce9a428841'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.141667 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] The token cache is being invalidate because project 3d0e10b4f3274ecbba8d23e9a0e4e3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:31.144364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.144364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.144364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1214/2405] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:30 2022] DELETE /identity/v3/projects/3d0e10b4f3274ecbba8d23e9a0e4e3a6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:31.154065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:31.169578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d8db-2c1b-4855-9ecd-bb5924a60c63 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:31.190322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:31.240143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:31.244262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:31.244262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:31.244262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] PATH_INFO: `/v3/users/83eb6c5bcd3741c395f2f9686518eaf7` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:31.244262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83eb6c5bcd3741c395f2f9686518eaf7)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:31.245670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:31.478375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83eb6c5bcd3741c395f2f9686518eaf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.479052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83eb6c5bcd3741c395f2f9686518eaf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.575122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83eb6c5bcd3741c395f2f9686518eaf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.583579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83eb6c5bcd3741c395f2f9686518eaf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.597277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-677d9b14-f1e8-4944-afe0-1ca30b886d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83eb6c5bcd3741c395f2f9686518eaf7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:31.604720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1192/2406] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:31 2022] DELETE /identity/v3/users/83eb6c5bcd3741c395f2f9686518eaf7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:32.312432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:32.314428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:32.339286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bfae6-5beb-4327-8572-92ce9a428841 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:32.414084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:32.414726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.415002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.415271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] PATH_INFO: `/v3/projects/45a3a76953804bf1aaaafa65a1451b2d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.416975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a3a76953804bf1aaaafa65a1451b2d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:32.419012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:32.522232 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:32.541190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a3a76953804bf1aaaafa65a1451b2d', 'request_id': 'req-005e307b-5502-4f05-8fce-b93bb8de8d13'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:32.547870 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a3a76953804bf1aaaafa65a1451b2d', 'request_id': 'req-005e307b-5502-4f05-8fce-b93bb8de8d13'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:32.621346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a3a76953804bf1aaaafa65a1451b2d', 'request_id': 'req-005e307b-5502-4f05-8fce-b93bb8de8d13'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:32.633008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] The token cache is being invalidate because project 45a3a76953804bf1aaaafa65a1451b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:32.633299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:32.634282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-005e307b-5502-4f05-8fce-b93bb8de8d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:32.636172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1215/2407] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:32 2022] DELETE /identity/v3/projects/45a3a76953804bf1aaaafa65a1451b2d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:33:32.641246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.641748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.641971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.662770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:32.664073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:32.724455 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1193/2408] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:32.733589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.733915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.734190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.762186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:32.763412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:32.771953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1216/2409] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:32.780645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:32.791433 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f911d-d1ee-49d9-b991-b87d235c5551 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:32.796169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:32.796861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.797136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.797406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.798082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:33:32.798381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:32.799975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:32.813406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1194/2410] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:33:32.817874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:32.827182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ff39e-2c5e-4e31-99ab-49bdf2bf2a8a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:32.831964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:32.832799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.833188 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.833464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.834183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:32.836199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:32.916064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1217/2411] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:32 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:32.920689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:32.932145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f85db0-fdbb-443a-8046-e919e357d47d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:32.959683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:32.991555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:32.992080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:32.992362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:32.992637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:32.993272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:32.995212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.118807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1195/2412] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:32 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:33.122918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.133510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d09852-6756-4e0a-85d0-245949d30640 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.143661 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.144337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.144615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.145442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.146207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.147750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.158728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1218/2413] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:33.168035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.178938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89434200-8b24-4718-afa1-100c29199a37 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.184089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.184697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.184973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.185238 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.185906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.187496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21022a8-8525-4f66-8681-d9a3d01f3ab0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.198032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1196/2414] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:33.202376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.213367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10abcd-899e-4fe2-a016-975f797602a8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.218070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.218723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.219003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.219278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] PATH_INFO: `/v3/projects/59e4ab703cbc40c3a8af1541fc984f39/users/505b24409e06425eae8d712a85cbb470/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.235559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e4ab703cbc40c3a8af1541fc984f39, user_id=505b24409e06425eae8d712a85cbb470, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.238476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.312967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1219/2415] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:33 2022] PUT /identity/v3/projects/59e4ab703cbc40c3a8af1541fc984f39/users/505b24409e06425eae8d712a85cbb470/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:33.324075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.324359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.324633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.354479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] MFA Rules not processed for user `505b24409e06425eae8d712a85cbb470`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:33.355757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:33.424124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1197/2416] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:33.429450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.495195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a25897-3329-499b-bc37-377b7000a2a3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.500241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.500848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.501129 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.501401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.502077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.504552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.628307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1220/2417] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:33 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:33.634071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.646777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54b772-5821-4d45-bcef-554cd52a865d None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.706890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.707677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.707982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.708358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.709020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.711040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.812579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1198/2418] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:33 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:33.816715 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.827818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc623b7c-bfb9-4ea6-9b63-55384c15fa39 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.832779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.833277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.833556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.833866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.834505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.836098 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.847091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1221/2419] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:33.851373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.861456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711aeb44-bc05-4ac5-8e67-93b5cbde8233 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.866645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:33.867281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:33.867553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:33.867822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] PATH_INFO: `/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2/users/9f0f79cb345e453381a641eddd4fb795/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:33.884537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a769d3cb91b4f33a37eb91d5f6592f2, user_id=9f0f79cb345e453381a641eddd4fb795, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:33.887230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:33.960914 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1199/2420] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:33 2022] PUT /identity/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2/users/9f0f79cb345e453381a641eddd4fb795/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:33.964951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:33.975362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a99002-b30b-4736-89e1-b0e8318b02da admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:33.997573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:34.041181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:34.041826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:34.042110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:34.042386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:34.043029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:34.044721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5624353-8ac0-4884-bbcc-37a1400b211b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:34.056772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1222/2421] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:34.061754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:34.067934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:34.073307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3487-9527-4dd7-9f7b-00f8f4b53914 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:34.081398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:34.081398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:34.081398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:34.081398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] PATH_INFO: `/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2/users/9f0f79cb345e453381a641eddd4fb795/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:34.082031 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a769d3cb91b4f33a37eb91d5f6592f2, user_id=9f0f79cb345e453381a641eddd4fb795, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:34.085367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:34.096333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1200/2422] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:34 2022] PUT /identity/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2/users/9f0f79cb345e453381a641eddd4fb795/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:34.107168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:34.110059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:34.110059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:34.138900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] MFA Rules not processed for user `9f0f79cb345e453381a641eddd4fb795`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:34.140179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:34.214173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1223/2423] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:34.226064 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:34.227816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:34.293035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba6741-37ae-444a-a026-1a42c3913567 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:34.296888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:34.297496 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:34.297806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:34.298080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:34.307452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:34.308887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:34.318903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1201/2424] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:34.465058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:34.474856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323bc87-a673-4f97-ad16-ace75246886a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:34.478505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:34.479137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:34.479416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:34.479689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:34.538281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:34.539754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b19b02-8936-439d-b7a9-717a37f9c87f service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:34.549786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1224/2425] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:35.038859 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:35.112067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a736-7447-4f45-b964-e29e5f0d8d2e service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:35.117633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:35.118631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:35.118631 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:35.118849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] PATH_INFO: `/v3/users/505b24409e06425eae8d712a85cbb470` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:35.120050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=505b24409e06425eae8d712a85cbb470)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:35.122027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:35.303443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '505b24409e06425eae8d712a85cbb470'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:35.304108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '505b24409e06425eae8d712a85cbb470'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:35.387009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '505b24409e06425eae8d712a85cbb470'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:35.394473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '505b24409e06425eae8d712a85cbb470'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:35.407513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '505b24409e06425eae8d712a85cbb470'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:35.414138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1202/2426] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:35 2022] DELETE /identity/v3/users/505b24409e06425eae8d712a85cbb470 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:35.418726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:35.419010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:35.419315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:35.439149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:35.440355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:35.483004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:35.502064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1225/2427] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:33:35.512478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:35.513953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:35.527860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:35.572771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d9d62-a689-4115-93f4-55525b3b1d73 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:35.576278 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:35.576873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:35.577146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:35.577412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:35.587318 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:35.588750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:35.598195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1203/2428] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:36.023010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:36.024399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:36.041003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a13fe3-654e-4287-a535-1f6e5c9765f8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:36.053611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:36.054381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:36.054678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:36.055083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] PATH_INFO: `/v3/projects/59e4ab703cbc40c3a8af1541fc984f39` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:36.056357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e4ab703cbc40c3a8af1541fc984f39)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:36.058492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:36.212380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e4ab703cbc40c3a8af1541fc984f39', 'request_id': 'req-600af64e-867c-478e-9c86-10d94c33aee5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.218609 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e4ab703cbc40c3a8af1541fc984f39', 'request_id': 'req-600af64e-867c-478e-9c86-10d94c33aee5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.349580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e4ab703cbc40c3a8af1541fc984f39', 'request_id': 'req-600af64e-867c-478e-9c86-10d94c33aee5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.360886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] The token cache is being invalidate because project 59e4ab703cbc40c3a8af1541fc984f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:36.361167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.361834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.363468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1226/2429] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:36 2022] DELETE /identity/v3/projects/59e4ab703cbc40c3a8af1541fc984f39 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:36.369289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:36.379322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17116c21-3ac2-4776-8631-8d69838ed5cd service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:36.436757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:36.437389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:36.437679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:36.437960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] PATH_INFO: `/v3/users/9f0f79cb345e453381a641eddd4fb795` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:36.439089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0f79cb345e453381a641eddd4fb795)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:36.440987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:36.681524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0f79cb345e453381a641eddd4fb795'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.682288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0f79cb345e453381a641eddd4fb795'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.787948 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0f79cb345e453381a641eddd4fb795'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.795525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0f79cb345e453381a641eddd4fb795'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.807988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-b9102650-88f0-4600-9834-a04c1bb7f41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0f79cb345e453381a641eddd4fb795'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:36.814205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1204/2430] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:33:36 2022] DELETE /identity/v3/users/9f0f79cb345e453381a641eddd4fb795 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:37.408829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:37.411064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:37.428283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af64e-867c-478e-9c86-10d94c33aee5 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:37.487382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:37.488117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:37.488952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:37.489394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] PATH_INFO: `/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:37.490614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a769d3cb91b4f33a37eb91d5f6592f2)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:37.492536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:37.645927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a769d3cb91b4f33a37eb91d5f6592f2', 'request_id': 'req-e9346cf9-2d64-4f08-b93f-37898fef75b7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:37.652406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a769d3cb91b4f33a37eb91d5f6592f2', 'request_id': 'req-e9346cf9-2d64-4f08-b93f-37898fef75b7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:37.731972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a769d3cb91b4f33a37eb91d5f6592f2', 'request_id': 'req-e9346cf9-2d64-4f08-b93f-37898fef75b7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:37.739969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:37.744699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] The token cache is being invalidate because project 0a769d3cb91b4f33a37eb91d5f6592f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:33:37.744990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:37.745771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e9346cf9-2d64-4f08-b93f-37898fef75b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:33:37.747352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1227/2431] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:33:37 2022] DELETE /identity/v3/projects/0a769d3cb91b4f33a37eb91d5f6592f2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:37.753579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:37.754614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:37.754816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:37.774399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:37.775610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:37.806804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:37.839317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1205/2432] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:37.904037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:37.904301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:37.904576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:37.925698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:37.929381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:37.935829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1228/2433] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:37.940382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:37.950844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9666a-30d5-404a-9bd4-d44264fdff37 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:37.955445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:37.956073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:37.956376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:37.956647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:37.957873 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:33:37.958202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:37.960022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:37.972625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1206/2434] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:33:37.977520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:37.987268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6ffe6-e90c-461d-8c29-a31eb3a248ca None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:37.992776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:37.993385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:37.993666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:37.993968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:37.994596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:37.996501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.132330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1229/2435] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:33:37 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:38.136932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:38.148802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bafd1-2c68-447c-91cb-2353f4dadd90 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:38.207900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:38.209103 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.209452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.209726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.214319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:38.214319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.361647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1207/2436] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:33:38 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:33:38.362915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:38.374482 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b0d83-54f8-4681-bebf-fa9453aff323 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:38.379206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:38.379824 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.380100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.380404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.381754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:38.382692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.393541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1230/2437] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:38.401172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:38.413324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48507-e685-4c93-9d3a-90f1fad247dd admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:38.418385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:38.419001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.419280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.419553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.420206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:38.422100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406f790-ee76-404a-9ec3-eeccc13c0aa1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.440305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1208/2438] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:33:38.444903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:38.455898 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400206e-f96d-498f-91c4-e2d5e3777862 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:38.462527 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:38.472472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.472472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.472472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] PATH_INFO: `/v3/projects/2abe379a357a40d891cb1a719b37e8a7/users/f6438d1ab0c844b8acd3a75cd9b6b68a/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.482511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abe379a357a40d891cb1a719b37e8a7, user_id=f6438d1ab0c844b8acd3a75cd9b6b68a, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:38.485140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.584317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1231/2439] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:33:38 2022] PUT /identity/v3/projects/2abe379a357a40d891cb1a719b37e8a7/users/f6438d1ab0c844b8acd3a75cd9b6b68a/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:33:38.594498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2dc14-b130-4527-bffc-0b8f1cdcaf71 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.594787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2dc14-b130-4527-bffc-0b8f1cdcaf71 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.595059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2dc14-b130-4527-bffc-0b8f1cdcaf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.624745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-39f2dc14-b130-4527-bffc-0b8f1cdcaf71 None None] MFA Rules not processed for user `f6438d1ab0c844b8acd3a75cd9b6b68a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:38.625986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-39f2dc14-b130-4527-bffc-0b8f1cdcaf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:38.689771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1209/2440] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:33:38.702544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:38.704269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:38.764233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef24307-8b01-4e17-98ef-a5c463f88b97 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:38.768184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:38.768807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:38.769081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:38.769350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:38.779606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:38.781064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:38.791692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1232/2441] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:33:43.965675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:43.966277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:43.966794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:43.999750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:33:44.000976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:44.015794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:44.068765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1210/2442] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:33:44.221307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:33:44.223329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:33:44.302376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e13ea4-20cc-4a13-9d0a-c11611b5b4f4 service glance] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:33:44.308195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:33:44.308810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:33:44.311021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:33:44.311021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:33:44.319422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:33:44.321155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:33:44.330196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1233/2443] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:33:54.103349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=16 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:33:54.326509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=86,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:11.228395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:11.229960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:11.241706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fa93c-08b0-4382-86c2-68e99da8a895 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:11.245295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:11.245927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:11.246194 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:11.246460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:11.248148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:11.322048 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:11.323412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:11.334446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1211/2444] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:11.513256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:11.514879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:11.573561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1057a-8928-4ef6-b5eb-4558c78a882c service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:11.577013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:11.577610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:11.577914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:11.578183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:11.579904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:11.589328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:11.590923 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0b24e-9342-4e3a-b27e-5cc913283fdd service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:11.599858 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1234/2445] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:11.678565 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:11.687854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b557b-4b3b-4272-9bf7-064a15cbaeeb service glance] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:11.691465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:11.692057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:11.692332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:11.692612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:11.694906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:11.709136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:11.710614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:11.721648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1212/2446] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:14.364250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:14.364250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:14.364250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:14.384170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:14.385434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:14.393165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1235/2447] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:34:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:14.405916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:14.418899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d61a3-4e8e-4555-9478-c951b6bf8fd3 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:14.424835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:14.425407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:14.425684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:14.425987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:14.436478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:14.438536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:14.449373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1213/2448] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:34:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:34:23.121278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:23.122689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:23.133929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b7478-7915-4c98-b38b-914b499acda8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:23.137456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:23.138072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:23.138347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:23.138614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:23.140356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:23.157793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:23.157793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:23.168972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1236/2449] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:24.445073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=46 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:31.425137 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:31.426867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:31.492153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88052f8-50c4-44a1-b66a-5c691dfc49e9 service swift] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:31.495904 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:31.496935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:31.496935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:31.497143 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:31.498835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:31.514344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:31.515966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:31.530249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1214/2450] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:33.163955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=35 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:35.066168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:35.130369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48e1a3-38fe-4770-aa15-2be433fb7541 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:35.136155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:35.136791 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:35.137069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:35.137338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] PATH_INFO: `/v3/users/f6438d1ab0c844b8acd3a75cd9b6b68a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:35.138505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6438d1ab0c844b8acd3a75cd9b6b68a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:35.140478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:35.172267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:35.174500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:35.200492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3f042-dcb7-4249-a17c-b4aed395ce7c service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:35.211065 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:35.211929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:35.212472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:35.212968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:35.215726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:35.304992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:35.306671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef96b1-3e74-4e50-bb23-bb429e9111b1 service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:35.318466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6438d1ab0c844b8acd3a75cd9b6b68a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:35.319242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6438d1ab0c844b8acd3a75cd9b6b68a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:35.325875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1215/2451] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:34:35.326647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6438d1ab0c844b8acd3a75cd9b6b68a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:35.397976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6438d1ab0c844b8acd3a75cd9b6b68a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:35.398745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6438d1ab0c844b8acd3a75cd9b6b68a'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:35.418675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1237/2452] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:34:35 2022] DELETE /identity/v3/users/f6438d1ab0c844b8acd3a75cd9b6b68a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:34:35.423233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:35.423522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:35.423794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:35.452199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:35.453491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:35.519900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1216/2453] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:35.530289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:35.531795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:35.592028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd06e6-2afa-41d3-a5c6-bf1463422357 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:35.595737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:35.596343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:35.596614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:35.596876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:35.608398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:35.609885 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d5f62-2f34-4771-8574-39af775898b2 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:35.621743 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1238/2454] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:34:36.123794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:36.125959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:36.150192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578417ca-2cd3-4b96-b58e-3aafbb4c4e66 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:36.157955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:36.158765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.159142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.159454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] PATH_INFO: `/v3/projects/2abe379a357a40d891cb1a719b37e8a7` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.160704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abe379a357a40d891cb1a719b37e8a7)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:36.167293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:36.175286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:36.210837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.211397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.211648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.233404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:36.234675 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:36.254165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:36.295939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1239/2455] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:34:36.308966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:36.311556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:36.335254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abe379a357a40d891cb1a719b37e8a7', 'request_id': 'req-131f397b-c8b1-4563-b869-46899db62cfd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:36.343215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abe379a357a40d891cb1a719b37e8a7', 'request_id': 'req-131f397b-c8b1-4563-b869-46899db62cfd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:36.393849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb3a27-8781-4a62-987d-2331799083eb None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:36.457964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:36.458584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.458865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.459139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.516764 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:36.518540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b22f9-8667-4620-a7f6-b9f93a04d6fd service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:36.519466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abe379a357a40d891cb1a719b37e8a7', 'request_id': 'req-131f397b-c8b1-4563-b869-46899db62cfd'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:36.533237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1240/2456] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:34:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 21:34:36.540422 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] The token cache is being invalidate because project 2abe379a357a40d891cb1a719b37e8a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:34:36.540734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:36.541553 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-131f397b-c8b1-4563-b869-46899db62cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:34:36.543513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1217/2457] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:34:36 2022] DELETE /identity/v3/projects/2abe379a357a40d891cb1a719b37e8a7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:34:36.550347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.550347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.550347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.587425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:36.588794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:36.667663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1241/2458] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:36.669161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:36.678803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.678803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.678803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.699520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:36.700762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:36.712435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1218/2459] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:36.730324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:36.741456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec853f74-de2b-4624-a5a7-5752f5f41655 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:36.746449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:36.747078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.747379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.747637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.748269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:34:36.748560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:36.750352 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:36.763122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1242/2460] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:34:36.767374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:36.777265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d015bc-f684-4b07-9c77-88a6a950cca7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:36.782100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:36.782956 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:36.783225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:36.783487 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:36.784107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:36.786047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:36.926932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1219/2461] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:34:36 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:34:36.931653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:36.941942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed630a7-2eee-4b46-bd52-232f57f9b4a1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:37.004322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:37.004930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.005207 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.005477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.006331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:37.008990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:37.158395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1243/2462] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:34:36 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:34:37.168096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:37.178526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1f9e-e44b-479e-aeb3-9b883d136a2d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:37.183828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:37.184467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.184746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.185018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.185664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:37.187301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:37.198633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1220/2463] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:34:37.208420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:37.214064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b8d3b-e287-402c-a630-998ce1e2f37c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:37.219138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:37.219868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.220047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.220321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.220982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:37.222631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc7ad9-9da1-4a79-bad3-db38ce2d1019 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:37.234201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1244/2464] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:34:37.239509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:37.250175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128f39a-31a6-4663-b7db-96eb18f18fa5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:37.255177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:37.256319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.256319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.256513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] PATH_INFO: `/v3/projects/ed9e8bc2976f4e2a8f439f89440bdaa3/users/7984fc2a59704c10905656ae32dc68af/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.273697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9e8bc2976f4e2a8f439f89440bdaa3, user_id=7984fc2a59704c10905656ae32dc68af, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:37.276600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:37.467226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1221/2465] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:34:37 2022] PUT /identity/v3/projects/ed9e8bc2976f4e2a8f439f89440bdaa3/users/7984fc2a59704c10905656ae32dc68af/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:34:37.473244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a8944-d05c-42dc-9c08-1e09050f81c9 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.473703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a8944-d05c-42dc-9c08-1e09050f81c9 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.474171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a8944-d05c-42dc-9c08-1e09050f81c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.504432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7d8a8944-d05c-42dc-9c08-1e09050f81c9 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:37.506432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7d8a8944-d05c-42dc-9c08-1e09050f81c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:37.579894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1245/2466] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:37.590291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:37.592183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:37.662653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39a4a-af57-4c98-9390-ef1c1889fb8a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:37.666669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:37.667313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:37.667577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:37.667850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:37.677615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:37.679625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:37.690054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1222/2467] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:44.236112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:44.236657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:44.236657 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:44.284815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] MFA Rules not processed for user `7984fc2a59704c10905656ae32dc68af`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:44.286095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:44.296481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:44.378170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1246/2468] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:44.383380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:44.393804 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f194d-7020-4bd0-9c37-3e2293aa256b service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:44.400545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:44.401169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:44.402816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:44.402816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:44.402816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:44.410992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:44.572598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1223/2469] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:34:44 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:34:44.576837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:44.588894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4671e2-0e14-4eba-a266-e21c89b2f543 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:44.663490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:44.664193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:44.664420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:44.664691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:44.665336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:44.667756 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:44.817452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1247/2470] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:34:44 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:34:44.822572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:44.833423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb4f56-8473-4bef-823e-f5bb409e2f33 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:44.839217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:44.839871 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:44.840145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:44.840412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:44.841051 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:44.842682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:44.855437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1224/2471] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:34:44.863116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:44.874279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e60d1-a9de-4beb-b56e-e5b64b3267b9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:44.879709 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:44.880374 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:44.880664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:44.880997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] PATH_INFO: `/v3/projects/17302b86015842cfab74aacb763b0130/users/3d3e5daa8cb449c38e9e34d8434367ce/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:44.897337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17302b86015842cfab74aacb763b0130, user_id=3d3e5daa8cb449c38e9e34d8434367ce, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:44.900937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:45.011428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1248/2472] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:34:44 2022] PUT /identity/v3/projects/17302b86015842cfab74aacb763b0130/users/3d3e5daa8cb449c38e9e34d8434367ce/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:34:45.015573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:45.021145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:45.027345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568a3ba-1666-4ea6-8686-930c5d2b2fb3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:45.094810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:45.095522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:45.095712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:45.095983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:45.096640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:45.098453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba2799-786c-40e2-a95d-4df35a774a22 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:45.111172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1225/2473] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:34:45.115614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:45.127289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6cc41-c39b-4dc0-ba16-73c04e8e8a4a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:45.132248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:45.132924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:45.133231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:45.133506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] PATH_INFO: `/v3/projects/17302b86015842cfab74aacb763b0130/users/3d3e5daa8cb449c38e9e34d8434367ce/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:45.135694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=17302b86015842cfab74aacb763b0130, user_id=3d3e5daa8cb449c38e9e34d8434367ce, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:45.138501 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:45.150507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1249/2474] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:34:45 2022] PUT /identity/v3/projects/17302b86015842cfab74aacb763b0130/users/3d3e5daa8cb449c38e9e34d8434367ce/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:34:52.289170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8de59-e3ec-45a4-9212-939203698d7d None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:52.289895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8de59-e3ec-45a4-9212-939203698d7d None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:52.290492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8de59-e3ec-45a4-9212-939203698d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:52.339224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-28c8de59-e3ec-45a4-9212-939203698d7d None None] MFA Rules not processed for user `3d3e5daa8cb449c38e9e34d8434367ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:52.341534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28c8de59-e3ec-45a4-9212-939203698d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:52.487361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1226/2475] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:34:52.519384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:34:52.519384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:52.604778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:34:52.631679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f32b-7b76-433d-9079-f4787dc11fbc admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:34:52.641184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:34:52.642190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:52.642631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:52.643068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:52.718104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:34:52.719737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b60e6-d641-42c0-b015-0c2878a1f915 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:34:52.730828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1250/2476] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:34:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:34:56.604598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:34:56.605593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:34:56.606124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:34:56.628861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] MFA Rules not processed for user `7984fc2a59704c10905656ae32dc68af`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:34:56.630224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:34:56.637947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1227/2477] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:02.365573 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:02.366262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:02.366262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:02.394414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] MFA Rules not processed for user `e9450f3f7527406ab70fcf0b64cab787`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:02.394414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:02.483109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1251/2478] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:06.624945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=65 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:12.472641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=54 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:14.365148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:14.366610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:14.434604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b4cb8-a920-4ffa-92db-b24cbba1b7f5 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:14.438465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:14.439080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:14.439490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:14.439770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:14.441559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:14.506326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:14.508207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:14.518369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1228/2479] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:18.903973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:18.905821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:18.920607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd43a2-5be1-49c9-8613-c9f3f5e60445 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:18.925256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:18.925888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:18.926144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:18.926418 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] PATH_INFO: `/v3/users/2ca47c37cd2d4999afa598cac6daa2e7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:18.940906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca47c37cd2d4999afa598cac6daa2e7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:18.942959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:19.161671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca47c37cd2d4999afa598cac6daa2e7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.162413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca47c37cd2d4999afa598cac6daa2e7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.170029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca47c37cd2d4999afa598cac6daa2e7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.272420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca47c37cd2d4999afa598cac6daa2e7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.273925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-42207d3d-27d8-4917-9cea-b1c03e10fb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca47c37cd2d4999afa598cac6daa2e7'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.301722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1252/2480] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:18 2022] DELETE /identity/v3/users/2ca47c37cd2d4999afa598cac6daa2e7 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:19.623827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:19.635480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f20083-ac13-46d9-a763-1995e06821a8 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:19.702738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:19.703648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:19.704527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:19.704527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] PATH_INFO: `/v3/projects/d1e4957407e948fc909aadade7d19719` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:19.718358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e4957407e948fc909aadade7d19719)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:19.720739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:19.864431 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:19.870000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e4957407e948fc909aadade7d19719', 'request_id': 'req-a1572f80-a721-4a9d-ab22-f3067605e744'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:19.877523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e4957407e948fc909aadade7d19719', 'request_id': 'req-a1572f80-a721-4a9d-ab22-f3067605e744'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:20.003678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e4957407e948fc909aadade7d19719', 'request_id': 'req-a1572f80-a721-4a9d-ab22-f3067605e744'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:20.016892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] The token cache is being invalidate because project d1e4957407e948fc909aadade7d19719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:20.017176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:20.017854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a1572f80-a721-4a9d-ab22-f3067605e744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:20.019552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1229/2481] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:19 2022] DELETE /identity/v3/projects/d1e4957407e948fc909aadade7d19719 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:20.025928 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.026431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.026554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.047111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:20.048450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:20.109677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1253/2482] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:20.119076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.119361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.122159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.140464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:20.141786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:20.150148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1230/2483] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:20.154607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.164719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b07c1-461c-4ea6-9cdd-702695088a55 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.169492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.170138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.170420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.170689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.171372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:20.171670 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.173264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.186340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1254/2484] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:35:20.190945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.208145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1975-95ff-4ebd-87d1-55e8b31c1053 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.210039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.211155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.211432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.211698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.212332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.214292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.346028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1231/2485] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:20 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:20.351318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.362985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d431712-1921-4894-a4a4-0070a1824cd5 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.410540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:20.424613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.425403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.425708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.426006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.426638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.428563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.547584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1255/2486] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:20 2022] POST /identity/v3/users => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:20.552030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.563122 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cf2-d283-40cf-89c6-f06556f48492 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.567664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.568279 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.568538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.568810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.569444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.571256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.582005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1232/2487] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:35:20.586381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.596589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704f8169-a0e8-40c0-b4d2-4bbef0120c5d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.601863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.602649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.602920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.603189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.603825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.605411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d760479-f83e-4545-8fd4-e4df833780e3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.616319 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1256/2488] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:20.623549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.634331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19b160-6371-43a7-81de-ad4015b27386 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.639025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.639680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.639985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.640274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] PATH_INFO: `/v3/projects/7c9a2a0956194446a0bd3de657b3a7af/users/df0f2a319e194d21b734b5b5e724a9e8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.656160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9a2a0956194446a0bd3de657b3a7af, user_id=df0f2a319e194d21b734b5b5e724a9e8, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.659142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.741445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1233/2489] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:20 2022] PUT /identity/v3/projects/7c9a2a0956194446a0bd3de657b3a7af/users/df0f2a319e194d21b734b5b5e724a9e8/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:20.746698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2b59-35e8-4c45-bd9f-f46a8cb383e4 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.746983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2b59-35e8-4c45-bd9f-f46a8cb383e4 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.747259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b2b59-35e8-4c45-bd9f-f46a8cb383e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.767567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-e32b2b59-35e8-4c45-bd9f-f46a8cb383e4 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:20.768763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e32b2b59-35e8-4c45-bd9f-f46a8cb383e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:20.827629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1257/2490] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:20.838272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:20.840025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:20.905061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0edb9a-7229-44f9-95c1-270036ce316d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:20.908950 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:20.909559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:20.909865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:20.910136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:20.919790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:20.921207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:20.932354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1234/2491] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:22.702346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:22.702346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:22.703750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:22.744955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] MFA Rules not processed for user `df0f2a319e194d21b734b5b5e724a9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:22.745312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:22.860321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1258/2492] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:22.861835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:22.873469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895f476-a5c8-45b3-bdff-7c21b8722a16 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:22.878195 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:22.878794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:22.879066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:22.879330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:22.879955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:22.881900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.016842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1235/2493] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:22 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:23.024737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:23.037778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41471f1-c378-4547-8be8-03fa11f29b4b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:23.098360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:23.110959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:23.111708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:23.111994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:23.112261 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:23.112894 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:23.115039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.223921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1259/2494] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:23 2022] POST /identity/v3/users => generated 309 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:23.235214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:23.239292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443c6f-3d43-4b98-9f2b-88338d10b1bf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:23.244120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:23.244808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:23.245010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:23.245260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:23.245931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:23.247604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.259906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1236/2495] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:23.264445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:23.277787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afc44-c28c-4449-a486-c04d92c6e6b2 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:23.282864 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:23.283821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:23.284101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:23.284372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] PATH_INFO: `/v3/projects/3e60ec7f07234e9ea3636d78ff167360/users/63662975bd9e4d43a6bc1a71d38d8af9/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:23.301229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e60ec7f07234e9ea3636d78ff167360, user_id=63662975bd9e4d43a6bc1a71d38d8af9, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:23.303932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.475785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1260/2496] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:23 2022] PUT /identity/v3/projects/3e60ec7f07234e9ea3636d78ff167360/users/63662975bd9e4d43a6bc1a71d38d8af9/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:35:23.489973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:23.496112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a212d-fd46-40cc-8a4d-e787f5bea3ae admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:23.537652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:23.561633 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:23.562274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:23.562581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:23.562828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:23.563466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:23.565059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd885e-4fd6-4de8-b51d-af52e9c48b77 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.575966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1237/2497] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:23.580447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:23.591978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323426-4f9d-4ac1-884c-159a6e454475 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:23.597183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:23.597877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:23.602882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:23.602882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] PATH_INFO: `/v3/projects/3e60ec7f07234e9ea3636d78ff167360/users/63662975bd9e4d43a6bc1a71d38d8af9/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:23.602882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e60ec7f07234e9ea3636d78ff167360, user_id=63662975bd9e4d43a6bc1a71d38d8af9, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:23.631932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:23.643488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1261/2498] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:23 2022] PUT /identity/v3/projects/3e60ec7f07234e9ea3636d78ff167360/users/63662975bd9e4d43a6bc1a71d38d8af9/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:25.306760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:25.307315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:25.307315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:25.341545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] MFA Rules not processed for user `63662975bd9e4d43a6bc1a71d38d8af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:25.342870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:25.422293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1238/2499] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:26.893131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:26.901443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:26.981400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d08fb-4839-4101-ac9d-8f12b4b820b3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:26.986737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:26.994588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:26.994588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:26.994588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:26.994588 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:27.002452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:27.004169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:27.015361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1262/2500] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:27.070411 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:27.176351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfa01c-8f6d-4fc5-9326-f94f100b0e6f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:27.184261 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:27.184701 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:27.185053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:27.185418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] PATH_INFO: `/v3/users/df0f2a319e194d21b734b5b5e724a9e8` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:27.187263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0f2a319e194d21b734b5b5e724a9e8)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:27.189554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:27.580860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0f2a319e194d21b734b5b5e724a9e8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:27.581559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0f2a319e194d21b734b5b5e724a9e8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:27.719678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0f2a319e194d21b734b5b5e724a9e8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:27.729023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:27.731857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0f2a319e194d21b734b5b5e724a9e8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:27.745236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0f2a319e194d21b734b5b5e724a9e8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:27.752057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1239/2501] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:27 2022] DELETE /identity/v3/users/df0f2a319e194d21b734b5b5e724a9e8 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:28.127116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:28.150259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de8754-4f9f-4aa3-9db1-c8d4709a7c4e service placement] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:28.226480 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:28.227065 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:28.227360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:28.227862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] PATH_INFO: `/v3/projects/7c9a2a0956194446a0bd3de657b3a7af` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:28.229179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9a2a0956194446a0bd3de657b3a7af)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:28.231400 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:28.271125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:28.274207 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:28.347290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc8a8f-28e2-453d-923c-674534949026 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:28.351464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:28.352184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:28.352548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:28.352949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:28.354820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:28.366203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:28.369524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:28.388936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1240/2502] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:28.416490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:28.425361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9a2a0956194446a0bd3de657b3a7af', 'request_id': 'req-20e205f6-b0ef-4ab3-a317-772970102102'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:28.436517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9a2a0956194446a0bd3de657b3a7af', 'request_id': 'req-20e205f6-b0ef-4ab3-a317-772970102102'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:28.615594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9a2a0956194446a0bd3de657b3a7af', 'request_id': 'req-20e205f6-b0ef-4ab3-a317-772970102102'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:28.637521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] The token cache is being invalidate because project 7c9a2a0956194446a0bd3de657b3a7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:28.637991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:28.638882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:28.640712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1263/2503] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:28 2022] DELETE /identity/v3/projects/7c9a2a0956194446a0bd3de657b3a7af => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:28.644993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:28.646644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:28.669799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe1d1-40a2-4128-87c2-69790db0c655 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:28.754611 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:28.755276 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:28.755627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:28.755966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] PATH_INFO: `/v3/users/63662975bd9e4d43a6bc1a71d38d8af9` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:28.757217 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63662975bd9e4d43a6bc1a71d38d8af9)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:28.760215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:29.061776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63662975bd9e4d43a6bc1a71d38d8af9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:29.062459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63662975bd9e4d43a6bc1a71d38d8af9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:29.181040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63662975bd9e4d43a6bc1a71d38d8af9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:29.188906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63662975bd9e4d43a6bc1a71d38d8af9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:29.203205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63662975bd9e4d43a6bc1a71d38d8af9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:29.209698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1241/2504] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:28 2022] DELETE /identity/v3/users/63662975bd9e4d43a6bc1a71d38d8af9 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:29.277998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:29.281167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:29.381378 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e205f6-b0ef-4ab3-a317-772970102102 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:29.495814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:29.496958 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:29.497531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:29.497987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:29.501118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:29.517071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:29.519317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14300c06-e82b-4d38-baf3-c13eeb2e9457 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:29.534594 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1264/2505] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:29.860262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:29.861824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:29.957158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2569d-b867-47d6-9446-7d96c15da5d6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:29.962985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:29.967502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:29.967502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:29.967502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] PATH_INFO: `/v3/projects/3e60ec7f07234e9ea3636d78ff167360` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:29.967502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e60ec7f07234e9ea3636d78ff167360)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:29.970414 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.260314 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e60ec7f07234e9ea3636d78ff167360', 'request_id': 'req-ac2be70e-a8e5-4d98-a285-0a3231048c82'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:30.267055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e60ec7f07234e9ea3636d78ff167360', 'request_id': 'req-ac2be70e-a8e5-4d98-a285-0a3231048c82'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:30.356927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e60ec7f07234e9ea3636d78ff167360', 'request_id': 'req-ac2be70e-a8e5-4d98-a285-0a3231048c82'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:30.369129 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] The token cache is being invalidate because project 3e60ec7f07234e9ea3636d78ff167360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:30.369420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:30.370097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-ac2be70e-a8e5-4d98-a285-0a3231048c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:30.371629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1242/2506] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:29 2022] DELETE /identity/v3/projects/3e60ec7f07234e9ea3636d78ff167360 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:30.376546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.377058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.377159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.398817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:30.400067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:30.468644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1265/2507] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:30.483898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.483898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.483898 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.501132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:30.502481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:30.510664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1243/2508] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:30.514964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.524644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df34a69-b86f-4333-b1c9-557013125aab None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:30.529420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:30.530077 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.530358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.530628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.531268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:30.531568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:30.533436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.545595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1266/2509] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:35:30.550085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.564851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd0751-b80c-4619-9c36-8b9d59751c09 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:30.570695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:30.571328 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.571607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.571884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.572519 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:30.574668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.723562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1244/2510] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:30 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:35:30.724867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.736141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24683677-9602-497a-af77-29c7d2ef0a69 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:30.796020 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:30.796686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.796926 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.797199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.797875 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:30.799821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.905578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1267/2511] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:30 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:30.907071 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.918939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210d1d-d220-48fa-afc3-a1b698b6f616 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:30.924097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:30.924764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.925108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.925969 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.926116 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:30.927748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.944663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1245/2512] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:30.950191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.957993 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:30.960629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be67d-d10e-4833-967e-18636a061b82 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:30.965382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:30.966009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:30.966282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:30.966608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:30.967218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:30.968807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd118c6f-b7be-471c-b179-f473c2a42262 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:30.979753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1268/2513] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:30.984146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:30.999895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb5ece-55b7-416e-8563-35eb03dacc27 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.000130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.000702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.000985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.001260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] PATH_INFO: `/v3/projects/65a74f9bc7a44aaaba5d950a3a26d8d5/users/eb9a7e0a20214ba1b67a808e3afa4910/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.017892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a74f9bc7a44aaaba5d950a3a26d8d5, user_id=eb9a7e0a20214ba1b67a808e3afa4910, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.020567 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ca302-aa5a-4d5d-94ff-e9be71dae692 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.039362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.039619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.040039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.059707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] MFA Rules not processed for user `7d2ee46e8302468bb90ee913eeeba4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:31.061044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:31.122118 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1246/2514] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:30 2022] PUT /identity/v3/projects/65a74f9bc7a44aaaba5d950a3a26d8d5/users/eb9a7e0a20214ba1b67a808e3afa4910/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:31.132566 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.133022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.137400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.138561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1269/2515] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:35:31.170259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] MFA Rules not processed for user `eb9a7e0a20214ba1b67a808e3afa4910`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:31.171839 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:31.239903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1247/2516] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:35:31.245382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.260784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bdf69-d579-414f-aae2-98e1a0892298 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.285323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:31.341568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.344633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.344633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.344633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.344633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.345260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.456681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.459547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1270/2517] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:31 2022] POST /identity/v3/projects => generated 369 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:31.488967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02158149-499a-4af5-95ce-d913d0350ac2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.562964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.562964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.562964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.562964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.563386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.564681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.712234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.737241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:31.737406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1248/2518] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:31 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:31.737406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.737406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16b72-3708-40bd-9db0-a4559f3491d4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.738689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.739284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.739561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.739831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.740511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.742396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.752497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1249/2519] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:35:31.756594 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.766988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49558b00-e2b8-44e3-9bba-752f4ed7e621 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.771739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.771968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4d140-d343-4a46-a348-1188ded9cdfa admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:31.772427 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.772709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.773563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] PATH_INFO: `/v3/projects/5cb8db482dbd41298f309e940f496e95/users/4ab5701a44914b308d331bb9decdc3b0/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.777046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:31.777665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:31.777978 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:31.778251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:31.780047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:31.790563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb8db482dbd41298f309e940f496e95, user_id=4ab5701a44914b308d331bb9decdc3b0, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.790800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:31.792147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.793385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:31.800856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1271/2520] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:35:31.923987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1250/2521] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:31 2022] PUT /identity/v3/projects/5cb8db482dbd41298f309e940f496e95/users/4ab5701a44914b308d331bb9decdc3b0/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:35:31.928205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:31.939828 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f0bf-7610-43dd-a131-ae26a9aa58df service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:32.022848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:32.024034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:32.024348 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:32.024616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:32.025268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:32.026937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b392c-5717-42e0-b7f9-2e33aaaa58c9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:32.043731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1272/2522] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:32.044840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:32.055714 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bd9a2-f5ea-4a9e-a31b-0c5c36a7800c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:32.060832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:32.061526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:32.061848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:32.062153 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] PATH_INFO: `/v3/projects/5cb8db482dbd41298f309e940f496e95/users/4ab5701a44914b308d331bb9decdc3b0/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:32.064091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb8db482dbd41298f309e940f496e95, user_id=4ab5701a44914b308d331bb9decdc3b0, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:32.066925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:32.078095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1251/2523] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:32 2022] PUT /identity/v3/projects/5cb8db482dbd41298f309e940f496e95/users/4ab5701a44914b308d331bb9decdc3b0/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:32.090193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:32.090481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:32.090760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:32.121431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] MFA Rules not processed for user `4ab5701a44914b308d331bb9decdc3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:32.122707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:32.195030 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1273/2524] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:32.206911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:32.208605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:32.258263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:32.260536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:32.294057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962fda00-b68a-4c5f-bce6-47ec29716bf6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:32.298192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:32.298816 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:32.299310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:32.299651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:32.315520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:32.318575 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:32.329620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1252/2525] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:32.362157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1b7b-21de-43db-ad5b-01b250a0dc34 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:32.369559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:32.370303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:32.370585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:32.370918 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:32.372751 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:32.386578 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:32.388401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fdfb-b85f-416a-8e59-df66e7de4985 service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:32.399364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1274/2526] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:32.999037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:33.580984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:35.055976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:35.274464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:36.231083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:36.306446 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c015d62-b9fc-4979-90e9-11bbdd40c142 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:36.311112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:36.311818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:36.312188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:36.312530 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] PATH_INFO: `/v3/users/eb9a7e0a20214ba1b67a808e3afa4910` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:36.313863 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9a7e0a20214ba1b67a808e3afa4910)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:36.315880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:36.598929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9a7e0a20214ba1b67a808e3afa4910'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:36.599751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9a7e0a20214ba1b67a808e3afa4910'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:36.706033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9a7e0a20214ba1b67a808e3afa4910'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:36.717543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9a7e0a20214ba1b67a808e3afa4910'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:36.735643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9a7e0a20214ba1b67a808e3afa4910'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:36.759543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1253/2527] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:36 2022] DELETE /identity/v3/users/eb9a7e0a20214ba1b67a808e3afa4910 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:36.761490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:36.762018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:36.762326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:36.797574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:36.799007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:36.875288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1275/2528] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:36.886754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:36.888253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:36.953878 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449ef7c-3457-4740-af7f-c7534749d2a8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:36.958529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:36.959445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:36.959690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:36.960098 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:36.970395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:36.972005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:36.988055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1254/2529] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:37.507822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:37.512119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:37.523788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dbd07-de8c-474f-90fa-e12b1630f940 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:37.528503 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:37.532124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:37.532124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:37.532124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] PATH_INFO: `/v3/projects/65a74f9bc7a44aaaba5d950a3a26d8d5` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:37.532577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a74f9bc7a44aaaba5d950a3a26d8d5)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:37.535242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:37.676807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a74f9bc7a44aaaba5d950a3a26d8d5', 'request_id': 'req-e89827ed-5e59-42b7-ab85-021dc2bcb050'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:37.687114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a74f9bc7a44aaaba5d950a3a26d8d5', 'request_id': 'req-e89827ed-5e59-42b7-ab85-021dc2bcb050'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:37.769453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a74f9bc7a44aaaba5d950a3a26d8d5', 'request_id': 'req-e89827ed-5e59-42b7-ab85-021dc2bcb050'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:37.786534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] The token cache is being invalidate because project 65a74f9bc7a44aaaba5d950a3a26d8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:37.787063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:37.787974 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:37.790155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1276/2530] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:37 2022] DELETE /identity/v3/projects/65a74f9bc7a44aaaba5d950a3a26d8d5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:37.794243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:37.812753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fd07d-7bed-410e-94f5-98bb6a93e6a6 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:37.835705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:37.898367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:37.899163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:37.899515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:37.899926 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] PATH_INFO: `/v3/users/4ab5701a44914b308d331bb9decdc3b0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:37.902574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab5701a44914b308d331bb9decdc3b0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:37.907441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:38.295309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab5701a44914b308d331bb9decdc3b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:38.296041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab5701a44914b308d331bb9decdc3b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:38.420405 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab5701a44914b308d331bb9decdc3b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:38.427923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab5701a44914b308d331bb9decdc3b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:38.448392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-642f902c-0870-4fa8-bd52-f21f345b4598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab5701a44914b308d331bb9decdc3b0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:38.457646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1255/2531] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:37 2022] DELETE /identity/v3/users/4ab5701a44914b308d331bb9decdc3b0 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:39.067362 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.068924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:39.083803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89827ed-5e59-42b7-ab85-021dc2bcb050 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.118510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:39.159537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.160326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.160599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.160869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] PATH_INFO: `/v3/projects/5cb8db482dbd41298f309e940f496e95` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.162043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb8db482dbd41298f309e940f496e95)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.164000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.284271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb8db482dbd41298f309e940f496e95', 'request_id': 'req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:39.290993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb8db482dbd41298f309e940f496e95', 'request_id': 'req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:39.351393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb8db482dbd41298f309e940f496e95', 'request_id': 'req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:39.362932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] The token cache is being invalidate because project 5cb8db482dbd41298f309e940f496e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:39.363859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:39.364528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a6fc6d31-39d5-4e5d-aebf-68f6a1624d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:39.366074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1277/2532] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:39 2022] DELETE /identity/v3/projects/5cb8db482dbd41298f309e940f496e95 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:39.372773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.373295 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.373637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.395203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:39.396415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:39.464918 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1256/2533] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:35:39.470571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.470944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.471308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.495953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:39.497367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:39.509845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1278/2534] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:39.513517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.524043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc718-31b2-4856-9081-976f70c696db None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.528801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.529409 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.529686 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.529985 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.530798 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:39.530954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.532612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.546164 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1257/2535] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:35:39.554068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.564541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e816cd-ad73-45be-8903-d2ea7713f147 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.569738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.570080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.570579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.570891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.571531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.573438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.720146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1279/2536] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:39 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:39.724537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.735764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3ff58-1802-4dfc-924d-c4137a3b4496 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.764744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:39.799023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.799962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.800322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.800668 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.801377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.803448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.885796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1258/2537] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:39 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:39.890336 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.916357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791ae5e-ea9d-41cc-9090-617a6901d5cd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.921231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.922021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.922300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.922571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.923249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.924913 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.936144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1280/2538] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:35:39.940178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.950128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b07b9-d400-4d0e-a8ad-618281ae5242 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.954819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.955419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.955711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.955959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:39.956593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:39.958249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e79164-a063-4725-a79f-28b9cf5d72e1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:39.970911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1259/2539] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:39.979216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:39.989390 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bc78b-cf6d-4a85-ba86-8883f3dd06c9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:39.994406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:39.995051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:39.995330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:39.995699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] PATH_INFO: `/v3/projects/d0f80ab3b2b14a288503ffe49ba88c57/users/258667cc77a84ad9b73c513f49008f71/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.011257 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f80ab3b2b14a288503ffe49ba88c57, user_id=258667cc77a84ad9b73c513f49008f71, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.013992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.095007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1281/2540] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:39 2022] PUT /identity/v3/projects/d0f80ab3b2b14a288503ffe49ba88c57/users/258667cc77a84ad9b73c513f49008f71/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:40.111620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.111905 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.112204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.142107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] MFA Rules not processed for user `258667cc77a84ad9b73c513f49008f71`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:40.144313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:40.209587 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1260/2541] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:40.214708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.267416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:40.278624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d01ab-5112-46db-8a94-ebd2ee91bc27 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.283844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.284450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.284729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.285000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.285868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.290642 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.451967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1282/2542] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:40 2022] POST /identity/v3/projects => generated 381 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:40.456289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.467438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554ce96-d8a3-4591-be1e-cad0dc0d2d31 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.525796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.526419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.526698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.526975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.527661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.529623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.638777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1261/2543] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:40 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:40.643787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.654802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c37a0-9971-4227-bfad-1783b2c3b591 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.659807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.660407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.660681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.660950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.661621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.663291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.685683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1283/2544] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:40.690001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.700766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fae34-145a-48be-b37e-038a2cd3b05d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.705529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.706327 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.706610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.706887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] PATH_INFO: `/v3/projects/f840465139244cfebb6499496220a4cd/users/09991cd8707f4b348862e599f3608054/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.722996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f840465139244cfebb6499496220a4cd, user_id=09991cd8707f4b348862e599f3608054, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.725678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.855691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1262/2545] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:40 2022] PUT /identity/v3/projects/f840465139244cfebb6499496220a4cd/users/09991cd8707f4b348862e599f3608054/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:40.859771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.870909 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596c98a-5f43-4e0f-9419-161706c1b828 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.930147 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.930758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.931031 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.931300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.932050 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.934934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f16c57-eafe-43d6-bdd4-b109d89bd87f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.943794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1284/2546] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:40.948324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:40.953776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:40.959093 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf239-cbe2-488e-b638-44e509defb53 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:40.963820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:40.968145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.968145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.968145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] PATH_INFO: `/v3/projects/f840465139244cfebb6499496220a4cd/users/09991cd8707f4b348862e599f3608054/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:40.968145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f840465139244cfebb6499496220a4cd, user_id=09991cd8707f4b348862e599f3608054, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:40.969557 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:40.981568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1263/2547] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:40 2022] PUT /identity/v3/projects/f840465139244cfebb6499496220a4cd/users/09991cd8707f4b348862e599f3608054/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:40.994363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:40.994655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:40.994931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:41.024639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] MFA Rules not processed for user `09991cd8707f4b348862e599f3608054`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:41.025892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:41.102359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1285/2548] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:41.114981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:41.117534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:41.178068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62374d16-353e-4f6d-9575-9c896f382c33 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:41.182317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:41.182923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:41.183204 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:41.183479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:41.192967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:41.194436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:41.204120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1264/2549] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:42.497663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:42.506959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd00a-c3d5-4bd3-baff-9d8fbda038cc None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:42.511998 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09991cd8707f4b348862e599f3608054', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f840465139244cfebb6499496220a4cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:42.519651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:42.519651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:42.519651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:42.519651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:42.519651 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:42.520102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:42.529057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1286/2550] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:35:42.533441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:42.544711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41322a2-902a-4a0f-9dae-1d6adca1cda9 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:42.549291 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09991cd8707f4b348862e599f3608054', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f840465139244cfebb6499496220a4cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:42.549911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:42.550189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:42.550509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:42.551149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:42.553034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:42.727448 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1265/2551] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:42 2022] POST /identity/v3/projects => generated 353 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:42.731932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:42.742555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb3a70-6bd1-4801-b893-efe5d8553736 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:42.778615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:42.810028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09991cd8707f4b348862e599f3608054', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f840465139244cfebb6499496220a4cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:42.810666 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:42.810941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:42.811210 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:42.811854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:42.812151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:42.813752 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:42.826879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1287/2552] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:35:42.867952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:42.878923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675ad65-7711-4199-942a-9bd3537672ab tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:42.883706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09991cd8707f4b348862e599f3608054', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f840465139244cfebb6499496220a4cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:42.884330 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:42.884612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:42.884913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] PATH_INFO: `/v3/projects/e0799882e4c741d7b1dfdf90ac49df83` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:42.886075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e0799882e4c741d7b1dfdf90ac49df83)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:42.888077 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:43.021979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0799882e4c741d7b1dfdf90ac49df83', 'request_id': 'req-cbb05f63-008c-4980-a69b-6307fb49d641'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:43.029582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0799882e4c741d7b1dfdf90ac49df83', 'request_id': 'req-cbb05f63-008c-4980-a69b-6307fb49d641'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:43.129245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0799882e4c741d7b1dfdf90ac49df83', 'request_id': 'req-cbb05f63-008c-4980-a69b-6307fb49d641'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:43.141282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] The token cache is being invalidate because project e0799882e4c741d7b1dfdf90ac49df83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:43.141570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:43.142243 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cbb05f63-008c-4980-a69b-6307fb49d641 tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:43.143725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1266/2553] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:42 2022] DELETE /identity/v3/projects/e0799882e4c741d7b1dfdf90ac49df83 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:43.901888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:43.903917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:43.982432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231813f-b910-4cfa-b203-59239689c31c tempest-VolumeQuotaClassesTest-1905910999 tempest-VolumeQuotaClassesTest-1905910999-project-admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:44.041381 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:44.042059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:44.042342 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:44.042614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] PATH_INFO: `/v3/users/258667cc77a84ad9b73c513f49008f71` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:44.043765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258667cc77a84ad9b73c513f49008f71)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:44.045691 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:44.274690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258667cc77a84ad9b73c513f49008f71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:44.275356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258667cc77a84ad9b73c513f49008f71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:44.283384 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258667cc77a84ad9b73c513f49008f71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:44.363233 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258667cc77a84ad9b73c513f49008f71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:44.363844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258667cc77a84ad9b73c513f49008f71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:44.376466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:44.382862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1288/2554] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:43 2022] DELETE /identity/v3/users/258667cc77a84ad9b73c513f49008f71 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:44.387652 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:44.388146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:44.388146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:44.407774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:44.409637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:44.459961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:44.473222 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1267/2555] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:44.484326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:44.485794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:44.547153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343fdce-d0cf-4203-ab55-ccb37b93b5d3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:44.551042 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:44.551650 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:44.551932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:44.552200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:44.567198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:44.568639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:44.579245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1289/2556] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:44.922832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:44.935486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46e60-4c3c-4f67-96cf-4646dc6ed22a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:44.939078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:44.939684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:44.939961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:44.940230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:44.941977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:45.010057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:45.011512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:45.021156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1268/2557] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:45.066109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:45.067584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:45.081085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ba12f-5f4b-425f-b5c8-cc25971b1e29 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:45.086807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:45.087996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:45.088377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:45.088753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] PATH_INFO: `/v3/projects/d0f80ab3b2b14a288503ffe49ba88c57` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:45.092855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f80ab3b2b14a288503ffe49ba88c57)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:45.092855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:45.210746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f80ab3b2b14a288503ffe49ba88c57', 'request_id': 'req-f8852549-8afb-47ee-bcf3-24414d368ebe'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.217881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f80ab3b2b14a288503ffe49ba88c57', 'request_id': 'req-f8852549-8afb-47ee-bcf3-24414d368ebe'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.328982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f80ab3b2b14a288503ffe49ba88c57', 'request_id': 'req-f8852549-8afb-47ee-bcf3-24414d368ebe'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.340641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] The token cache is being invalidate because project d0f80ab3b2b14a288503ffe49ba88c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:45.341047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.341690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.343227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1290/2558] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:45 2022] DELETE /identity/v3/projects/d0f80ab3b2b14a288503ffe49ba88c57 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:45.347524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:45.358538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef933d-db13-4d9f-8de7-a34310693bd5 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:45.417415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:45.418568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:45.418568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:45.418787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] PATH_INFO: `/v3/users/09991cd8707f4b348862e599f3608054` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:45.419760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09991cd8707f4b348862e599f3608054)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:45.421732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:45.634875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09991cd8707f4b348862e599f3608054'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.635614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09991cd8707f4b348862e599f3608054'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.769047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09991cd8707f4b348862e599f3608054'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.777962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09991cd8707f4b348862e599f3608054'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.792208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09991cd8707f4b348862e599f3608054'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:45.799959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1269/2559] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:45 2022] DELETE /identity/v3/users/09991cd8707f4b348862e599f3608054 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:35:46.052440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:46.054712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:46.161426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8852549-8afb-47ee-bcf3-24414d368ebe admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:46.199579 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:46.283413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:46.284104 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:46.284478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:46.284814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:46.288029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:46.307413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:46.309459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb73332-cec1-4d9c-b104-8b4ece4d4903 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:46.325627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1291/2560] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:46.442858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:46.445162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:46.460172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:46.549100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8a80-c412-4414-8629-767d90b0c0d2 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:46.556623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:46.556623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:46.556623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:46.556623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] PATH_INFO: `/v3/projects/f840465139244cfebb6499496220a4cd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:46.557529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f840465139244cfebb6499496220a4cd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:46.559528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:46.734309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f840465139244cfebb6499496220a4cd', 'request_id': 'req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:46.745785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f840465139244cfebb6499496220a4cd', 'request_id': 'req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:46.906632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f840465139244cfebb6499496220a4cd', 'request_id': 'req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:46.920880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] The token cache is being invalidate because project f840465139244cfebb6499496220a4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:46.921407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:46.922875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-60a2af03-0e0b-4c3e-bc1c-449d48453a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:46.926539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1270/2561] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:46 2022] DELETE /identity/v3/projects/f840465139244cfebb6499496220a4cd => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:46.932554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:46.933126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:46.933485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:46.967512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:46.969057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:47.044470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1292/2562] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:47.053357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.053853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.054203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.077046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:47.078678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:47.087526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1271/2563] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:47.091997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.103435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44b675-a1c9-446d-8cb8-ad373cf33654 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.108457 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.109109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.109385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.109660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.110521 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:47.110833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.112466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.126994 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1293/2564] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:35:47.132292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.142416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9d586-c178-41ac-b1f3-dd2178a5a059 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.152504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.153577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.154096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.154618 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.155808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.159880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.344058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1272/2565] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:47 2022] POST /identity/v3/projects => generated 389 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:47.348441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.359852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893844e2-24f7-42db-8884-288a65113524 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.432613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.433396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.433798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.434164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.434947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.436984 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.570809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1294/2566] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:47 2022] POST /identity/v3/users => generated 295 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:47.574748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.588737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec93b3-154e-418c-86a6-ba3657733065 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.595872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.596498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.596776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.597076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.597746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.599395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.615777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1273/2567] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:47.619866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.631201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1cd440-df84-419a-9f63-82a9575defd7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.640195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.640195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.640195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.640195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.643385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.643385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.657776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1295/2568] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:47.662024 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.673683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0711f-5237-4737-9b6c-98e493a81e99 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.683434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.683434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.683434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.683434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] PATH_INFO: `/v3/projects/1e30df789952449a920f2092dd3f776f/users/65b773189d1a4d6494d82cfabcd4e2a5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.699599 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e30df789952449a920f2092dd3f776f, user_id=65b773189d1a4d6494d82cfabcd4e2a5, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.702973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf7c4-59ff-495e-b9e2-3782e27d5a17 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.749203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:47.758612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:47.781699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:47.855259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1274/2569] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:47 2022] PUT /identity/v3/projects/1e30df789952449a920f2092dd3f776f/users/65b773189d1a4d6494d82cfabcd4e2a5/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:35:47.856161 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7e162-efea-46e8-a0a5-3f7f2ffbcab8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:47.859501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.859870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.860343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.902810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] MFA Rules not processed for user `65b773189d1a4d6494d82cfabcd4e2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:47.904134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:47.932108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:47.932733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:47.933016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:47.933290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:47.934220 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:47.935165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:47.948690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:47.950631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:47.962358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1296/2570] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:35:47.986608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1275/2571] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:47.987841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.063786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1330-706b-409e-8ef2-c21eb2d3338e service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.075092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.075708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.075986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.076258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.076979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.079005 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.285598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1297/2572] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:47 2022] POST /identity/v3/projects => generated 389 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:48.291690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.303454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a82d9-cfb8-4e49-89cd-66337aaf6b18 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.370228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.370944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.371298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.371645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.372426 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.374656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.501752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1276/2573] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:48 2022] POST /identity/v3/users => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:48.506497 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.519024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380ae3e-6f25-4525-891a-bb09a50e37bc admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.526731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.531471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.531471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.531471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.531471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.531471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.549837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1298/2574] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:48.553746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.571078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a495-dcae-4938-adc9-c0bf5721fc83 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.574667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.574890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.575462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.575462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] PATH_INFO: `/v3/projects/c7814ff7786344a2aea34e4220281fdc/users/2b9a8727f14f4d63b7ca273ec0c97cbc/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.593232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7814ff7786344a2aea34e4220281fdc, user_id=2b9a8727f14f4d63b7ca273ec0c97cbc, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.595944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.684654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1277/2575] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:48 2022] PUT /identity/v3/projects/c7814ff7786344a2aea34e4220281fdc/users/2b9a8727f14f4d63b7ca273ec0c97cbc/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:48.689557 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.707558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c836e7-1310-45a3-98ee-33be2d4517e8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.713287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:48.771403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.772021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.772296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.772564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.773201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.775682 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c82795-b6f4-487a-a432-398a036d1385 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.786317 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1299/2576] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:35:48.790872 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:48.804130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33704b-8768-41bf-9c6f-8aa6ba5d184e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:48.811463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:48.812141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.812400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.812676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] PATH_INFO: `/v3/projects/c7814ff7786344a2aea34e4220281fdc/users/2b9a8727f14f4d63b7ca273ec0c97cbc/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.814819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7814ff7786344a2aea34e4220281fdc, user_id=2b9a8727f14f4d63b7ca273ec0c97cbc, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:48.817463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:48.840384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1278/2577] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:48 2022] PUT /identity/v3/projects/c7814ff7786344a2aea34e4220281fdc/users/2b9a8727f14f4d63b7ca273ec0c97cbc/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:35:48.842552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:48.842835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:48.843103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:48.897281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] MFA Rules not processed for user `2b9a8727f14f4d63b7ca273ec0c97cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:48.899420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:49.005815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1300/2578] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:49.016376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:49.019241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:49.089967 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d72e61-73a7-48b2-b89c-b29fdee6547f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:49.093604 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:49.094226 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:49.094501 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:49.094765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:49.105218 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:49.106706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996e3b8-323e-40cf-969c-90e6779193fb service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:49.116265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1279/2579] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:35:49.259191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:49.324601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bb8c5-a7d6-4fab-b827-12949c43df5a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:49.329195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:49.329856 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:49.330133 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:49.330403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] PATH_INFO: `/v3/users/65b773189d1a4d6494d82cfabcd4e2a5` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:49.331512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b773189d1a4d6494d82cfabcd4e2a5)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:49.333416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:49.623734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:49.643066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b773189d1a4d6494d82cfabcd4e2a5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:49.644417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b773189d1a4d6494d82cfabcd4e2a5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:49.654360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b773189d1a4d6494d82cfabcd4e2a5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:49.758489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b773189d1a4d6494d82cfabcd4e2a5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:49.760175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b773189d1a4d6494d82cfabcd4e2a5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:49.779332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1301/2580] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:49 2022] DELETE /identity/v3/users/65b773189d1a4d6494d82cfabcd4e2a5 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:49.784395 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:49.784845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:49.785047 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:49.808044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:49.809528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:49.822711 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:49.879840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1280/2581] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:49.891371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:49.892924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:49.971029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8da069-13ad-470e-b590-a6f2317110f9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:49.974905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:49.975587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:49.975862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:49.976132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:49.985620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:49.987081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:49.997608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1302/2582] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:35:50.720677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:50.723089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:50.760834 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1b4b-e400-491d-bd24-93d974580677 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:50.774472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:50.776533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:50.776951 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:50.778028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] PATH_INFO: `/v3/projects/1e30df789952449a920f2092dd3f776f` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:50.779473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e30df789952449a920f2092dd3f776f)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:50.782249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:50.979277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e30df789952449a920f2092dd3f776f', 'request_id': 'req-86b4f7ee-48b8-456b-beda-c8c23cc397d7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:50.987867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e30df789952449a920f2092dd3f776f', 'request_id': 'req-86b4f7ee-48b8-456b-beda-c8c23cc397d7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.089355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e30df789952449a920f2092dd3f776f', 'request_id': 'req-86b4f7ee-48b8-456b-beda-c8c23cc397d7'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.104191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] The token cache is being invalidate because project 1e30df789952449a920f2092dd3f776f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:51.104191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.104191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.104777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1281/2583] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:50 2022] DELETE /identity/v3/projects/1e30df789952449a920f2092dd3f776f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:51.109476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:51.126280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46433830-ec13-41b2-99f1-2be2ac5e6aa8 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:51.208953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:51.209873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:51.210157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:51.210433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] PATH_INFO: `/v3/users/2b9a8727f14f4d63b7ca273ec0c97cbc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:51.211677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9a8727f14f4d63b7ca273ec0c97cbc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:51.213957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:51.493641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9a8727f14f4d63b7ca273ec0c97cbc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.494363 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9a8727f14f4d63b7ca273ec0c97cbc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.501903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:51.504368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9a8727f14f4d63b7ca273ec0c97cbc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.631625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9a8727f14f4d63b7ca273ec0c97cbc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.632265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-58b55591-4a72-46a8-b75d-5264dcdf8f6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9a8727f14f4d63b7ca273ec0c97cbc'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:51.653412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1303/2584] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:35:51 2022] DELETE /identity/v3/users/2b9a8727f14f4d63b7ca273ec0c97cbc => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:52.147975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:52.150999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:52.169313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f7ee-48b8-456b-beda-c8c23cc397d7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:52.238870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:52.239497 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.239774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.240043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] PATH_INFO: `/v3/projects/c7814ff7786344a2aea34e4220281fdc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.241174 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7814ff7786344a2aea34e4220281fdc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:52.243920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:52.395622 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:52.400134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7814ff7786344a2aea34e4220281fdc', 'request_id': 'req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:52.406625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7814ff7786344a2aea34e4220281fdc', 'request_id': 'req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:52.500322 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7814ff7786344a2aea34e4220281fdc', 'request_id': 'req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:52.515337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] The token cache is being invalidate because project c7814ff7786344a2aea34e4220281fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:35:52.515789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:52.516469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f9cfb5e9-8629-4b99-8c70-1c97d418c7e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:35:52.518145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1282/2585] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:35:52 2022] DELETE /identity/v3/projects/c7814ff7786344a2aea34e4220281fdc => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:52.524074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.524567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.524663 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.549124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:52.550398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:52.631169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1304/2586] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:52.639715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.639988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.640256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.661934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:52.663066 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:52.676432 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1283/2587] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:52.680432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:52.690631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc588da8-3ffe-4b69-8c74-d888981a540a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:52.696117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:52.696735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.697014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.697290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.697987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:35:52.698327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:52.699904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:52.713668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1305/2588] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:35:52.718423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:52.730332 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6edc76-b7a3-42e5-9063-0e6eb1012f98 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:52.738283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:52.738283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.738577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.738896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.739760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:52.741749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:52.839160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1284/2589] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:35:52 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:35:52.845094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:52.860071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde0a82-03c5-498d-8bcc-8051203576ff admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:52.936347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:52.937313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:52.937593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:52.937925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:52.938613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:52.940589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:53.082044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1306/2590] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:35:52 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:35:53.083384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:53.096272 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81d27a-fe2b-4cb4-85a2-d3972a9c7cee admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:53.101197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:53.101959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:53.102240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:53.102514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:53.103166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:53.104822 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:53.118934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1285/2591] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:53.121012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:53.132405 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe6042-c142-446b-9d9b-11f9567de26d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:53.138036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:53.138860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:53.139199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:53.139976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:53.140293 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:53.142035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d484d-38f6-44cb-ac5c-09ecf0ba98d9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:53.156387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1307/2592] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:35:53.160682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:53.174110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11cb2-c3ec-475a-bbcf-354135476a4f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:53.181479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:53.182580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:53.183349 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:53.183479 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] PATH_INFO: `/v3/projects/8b4cb0fcf746423d893a463f1826b97b/users/a9c9215a987d482983ead148d79c04e4/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:53.201667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4cb0fcf746423d893a463f1826b97b, user_id=a9c9215a987d482983ead148d79c04e4, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:53.206562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:53.301838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:53.302163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:53.302413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:53.306428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1286/2593] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:35:53 2022] PUT /identity/v3/projects/8b4cb0fcf746423d893a463f1826b97b/users/a9c9215a987d482983ead148d79c04e4/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:35:53.478561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] MFA Rules not processed for user `a9c9215a987d482983ead148d79c04e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:35:53.479845 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:53.521906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:35:53.577376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1308/2594] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:35:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:35:53.590025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:53.608464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:53.652456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcac18-7a49-427e-9c64-401778250c3d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:53.656416 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:53.657060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:53.657445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:53.657613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:53.668079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:53.669550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:53.692015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1287/2595] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:35:57.520369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:35:57.521862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:57.580800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c1a1-357c-4e89-85d9-1b2459e13517 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:35:57.584520 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:35:57.585124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:35:57.585401 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:35:57.587428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:35:57.587707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:35:57.660299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:35:57.661745 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:35:57.672156 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1309/2596] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:01.393168 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:01.394692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:01.409692 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd95df-6f7a-48e4-b200-28d9176c536f service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:01.415058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:01.415678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:01.416372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:01.416653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] PATH_INFO: `/v3/users/e4f87633619b42d989942e1d63d30fb0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:01.418017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f87633619b42d989942e1d63d30fb0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:01.419928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:01.645744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f87633619b42d989942e1d63d30fb0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:01.646353 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f87633619b42d989942e1d63d30fb0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:01.779833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f87633619b42d989942e1d63d30fb0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:01.788507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f87633619b42d989942e1d63d30fb0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:01.791689 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:01.802740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-7c5e5fe6-1d56-48ce-b3ac-cfe214ab874d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f87633619b42d989942e1d63d30fb0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:01.809402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1288/2597] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:01 2022] DELETE /identity/v3/users/e4f87633619b42d989942e1d63d30fb0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:02.056408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.067025 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af531de-8d45-4e6d-813b-b80230c6f98b service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.139380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.140029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.140790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.140790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] PATH_INFO: `/v3/projects/94e62daccf4b442c8ff2971abbedd74f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.141861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e62daccf4b442c8ff2971abbedd74f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.143781 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.167337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:02.260770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e62daccf4b442c8ff2971abbedd74f', 'request_id': 'req-a2bd1f90-cb92-4575-aa7f-2befff293e81'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:02.267658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e62daccf4b442c8ff2971abbedd74f', 'request_id': 'req-a2bd1f90-cb92-4575-aa7f-2befff293e81'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:02.352465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e62daccf4b442c8ff2971abbedd74f', 'request_id': 'req-a2bd1f90-cb92-4575-aa7f-2befff293e81'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:02.364234 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] The token cache is being invalidate because project 94e62daccf4b442c8ff2971abbedd74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:02.364597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:02.365249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-a2bd1f90-cb92-4575-aa7f-2befff293e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:02.366746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1310/2598] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:02 2022] DELETE /identity/v3/projects/94e62daccf4b442c8ff2971abbedd74f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:02.373091 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.373374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.373645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.395355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:02.396551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:02.458203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1289/2599] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:02.466964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.467255 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.467535 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.487398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:02.488628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:02.497511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1311/2600] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:02.502021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.512489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4ad1b-1b8d-4009-a16b-996bf11dce0b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.517019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.517607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.517963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.518239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.518940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:02.519282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.520860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.533033 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1290/2601] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:02.542999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.552556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c1fa-d3e3-4eb7-9352-886a1cf95f25 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.558396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.559002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.559272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.559569 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.560225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.562205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.662852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1312/2602] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:02 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:02.667935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.678456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28b64-1fb1-46d2-bfa3-4a8f1e2d3f18 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.746216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.746838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.747121 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.747390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.748035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.750004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.843824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1291/2603] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:02 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:02.848028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.859495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df92d6-05a9-4a5e-8c47-ecc17d239f93 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.864307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.864910 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.865187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.865453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.866128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.868354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.879722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1313/2604] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:02.884166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.894608 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f2cb4-fd20-46ed-8e2e-11b60fc1ccbc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.900096 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.900704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.900983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.901254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.901919 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.903512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bf771-87d3-47e0-bbe0-45a517ef8abf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:02.915713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1292/2605] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:02.920082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:02.931704 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692ab47-09e3-4676-851c-6feb2e82a4d4 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:02.940702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:02.941304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:02.941582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:02.941886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] PATH_INFO: `/v3/projects/04f38556369640cebc37272ab9176f7f/users/0a0da23fd7724dbc82d97090aa32e207/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:02.958191 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f38556369640cebc37272ab9176f7f, user_id=0a0da23fd7724dbc82d97090aa32e207, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:02.961701 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.026602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1314/2606] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:02 2022] PUT /identity/v3/projects/04f38556369640cebc37272ab9176f7f/users/0a0da23fd7724dbc82d97090aa32e207/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:03.038418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.039351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.039532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.069333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] MFA Rules not processed for user `0a0da23fd7724dbc82d97090aa32e207`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:03.070598 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:03.232483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:03.260776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1293/2607] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:03.265438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.332117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465d1f1-b419-40d0-a33a-a5366a0e241f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.338333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.339004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.339277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.339554 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.340183 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.342229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.451083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1315/2608] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:03 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:03.455193 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.467165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f8be-7ccd-4c31-a334-2e1fa50e1965 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.524460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.525259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.525463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.525563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.526286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.528311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.638035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1294/2609] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:03 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:03.642712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.653954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08692d2-1afd-4ca1-8224-42d3c1ae2093 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.659016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.659611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.659886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.660157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.660789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.662463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.672987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1316/2610] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:03.678449 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.690642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e78522-a1ac-4eb1-9a7e-5ecaa83bf9b9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.696022 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.696671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.696947 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.697236 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] PATH_INFO: `/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69/users/b0b64c2cb2734e2085134b19573bb251/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.712382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceb74c249fa43cca4f6f1541adc2c69, user_id=b0b64c2cb2734e2085134b19573bb251, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.715082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.815582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1295/2611] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:03 2022] PUT /identity/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69/users/b0b64c2cb2734e2085134b19573bb251/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:03.819759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.831850 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d605f7-afce-4660-9479-db90f573cbc7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.844808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:03.892629 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.893244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.893541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.893860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.894544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.896149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5f0a3-2d9e-46a5-bfc8-1889db8f27c0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.908603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1317/2612] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:03.912678 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:03.923233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15dd3-20c4-4a9c-a88c-93c8a86aa6e7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:03.927977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:03.928635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.929134 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.929326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] PATH_INFO: `/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69/users/b0b64c2cb2734e2085134b19573bb251/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.932037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ceb74c249fa43cca4f6f1541adc2c69, user_id=b0b64c2cb2734e2085134b19573bb251, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:03.935617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:03.946348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1296/2613] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:03 2022] PUT /identity/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69/users/b0b64c2cb2734e2085134b19573bb251/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:03.957398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:03.957671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:03.957966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:03.989607 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] MFA Rules not processed for user `b0b64c2cb2734e2085134b19573bb251`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:03.990905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:04.061630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1318/2614] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:04.073702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:04.075638 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:04.140109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e7bc2-34e9-410a-ae9f-48a2a3ed7ccb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:04.144029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:04.144727 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:04.145119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:04.145548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:04.155857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:04.157315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:04.170388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1297/2615] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:04.414123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:04.424788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b0087-1931-46df-83e7-19c2ad94c860 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:04.428430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:04.429034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:04.429300 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:04.429563 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:04.489438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:04.490930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723ed80-da70-4e25-9281-1ce085aa7266 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:04.500931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1319/2616] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:10.472436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:10.506675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:10.535688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143f15c-3954-4b34-a76e-ec82c050c1e4 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:10.541002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:10.541826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:10.542107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:10.542376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] PATH_INFO: `/v3/users/0a0da23fd7724dbc82d97090aa32e207` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:10.543509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a0da23fd7724dbc82d97090aa32e207)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:10.545539 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:10.893877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a0da23fd7724dbc82d97090aa32e207'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:10.949423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a0da23fd7724dbc82d97090aa32e207'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:10.979848 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a0da23fd7724dbc82d97090aa32e207'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:10.987681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a0da23fd7724dbc82d97090aa32e207'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.000476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a0da23fd7724dbc82d97090aa32e207'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.006573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1298/2617] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:10 2022] DELETE /identity/v3/users/0a0da23fd7724dbc82d97090aa32e207 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:11.012324 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:11.012860 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:11.013369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:11.043160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:11.044373 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:11.076635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:11.104640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1320/2618] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:11.114177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:11.115614 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:11.174679 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77984f39-ff05-4786-8618-615a30e0edab admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:11.178157 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:11.178785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:11.179057 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:11.179323 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:11.189058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:11.190529 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:11.200810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1299/2619] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:11.683247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:11.683247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:11.690750 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1160-2f02-4e8f-927d-3bad2aa095fd None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:11.695882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:11.696630 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:11.697404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:11.697404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] PATH_INFO: `/v3/projects/04f38556369640cebc37272ab9176f7f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:11.698380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f38556369640cebc37272ab9176f7f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:11.700504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:11.861738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f38556369640cebc37272ab9176f7f', 'request_id': 'req-8865ad0d-9e68-4786-aabe-4a1478584001'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.871881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f38556369640cebc37272ab9176f7f', 'request_id': 'req-8865ad0d-9e68-4786-aabe-4a1478584001'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.968309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f38556369640cebc37272ab9176f7f', 'request_id': 'req-8865ad0d-9e68-4786-aabe-4a1478584001'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.980502 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] The token cache is being invalidate because project 04f38556369640cebc37272ab9176f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:11.980785 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.981485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:11.983702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1321/2620] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:11 2022] DELETE /identity/v3/projects/04f38556369640cebc37272ab9176f7f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:11.987769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:11.997660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3dcb-5c65-4ca7-bfd4-3379cb1f34c6 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:12.049418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:12.057045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:12.059179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:12.059179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:12.059179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] PATH_INFO: `/v3/users/b0b64c2cb2734e2085134b19573bb251` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:12.059525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0b64c2cb2734e2085134b19573bb251)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:12.061417 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:12.270304 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0b64c2cb2734e2085134b19573bb251'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:12.270877 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0b64c2cb2734e2085134b19573bb251'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:12.339912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0b64c2cb2734e2085134b19573bb251'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:12.349263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0b64c2cb2734e2085134b19573bb251'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:12.361870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-58a6aaba-9fc6-4b16-9435-5cbd200a4170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0b64c2cb2734e2085134b19573bb251'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:12.367995 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1300/2621] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:11 2022] DELETE /identity/v3/users/b0b64c2cb2734e2085134b19573bb251 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:12.810883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:12.812364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:12.827855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865ad0d-9e68-4786-aabe-4a1478584001 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:12.899385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:12.900339 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:12.900474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:12.900694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] PATH_INFO: `/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:12.901866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ceb74c249fa43cca4f6f1541adc2c69)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:12.903796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.067846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ceb74c249fa43cca4f6f1541adc2c69', 'request_id': 'req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:13.074185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ceb74c249fa43cca4f6f1541adc2c69', 'request_id': 'req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:13.179792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ceb74c249fa43cca4f6f1541adc2c69', 'request_id': 'req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:13.183608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:13.193032 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] The token cache is being invalidate because project 6ceb74c249fa43cca4f6f1541adc2c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:13.193327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:13.194019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8000377f-60d4-43e2-8a5e-9b1d01a94e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:13.195575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1322/2622] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:12 2022] DELETE /identity/v3/projects/6ceb74c249fa43cca4f6f1541adc2c69 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:13.201621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae568c-a334-4749-83ac-569baabefcba None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.202177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae568c-a334-4749-83ac-569baabefcba None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.202177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae568c-a334-4749-83ac-569baabefcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.223660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-62ae568c-a334-4749-83ac-569baabefcba None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:13.224934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-62ae568c-a334-4749-83ac-569baabefcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:13.299902 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1301/2623] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:13.305558 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.306515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.307536 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.333841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:13.333841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:13.340589 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1323/2624] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:13.344979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae568c-a334-4749-83ac-569baabefcba None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.356843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae568c-a334-4749-83ac-569baabefcba None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.362101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.362683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.362979 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.363251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.363913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:13.364214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.365867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.380506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1302/2625] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:13.386078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.395879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5953b2-065f-4885-87d6-3e6657683495 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.402254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.402886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.403162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.403427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.404113 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.406158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.508323 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1324/2626] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:13 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:13.513303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.523335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2f0bb-cdff-48f2-9f7d-87bd612d3ee6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.586321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.586838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.587640 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.588002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.588932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.590900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.600387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:13.723632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1303/2627] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:13 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:13.728786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.739608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a613c7-08de-400a-837d-b4ca4a63b61a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.744927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.745809 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.746142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.746477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.747277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.748924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.760074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1325/2628] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:36:13.765762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.776972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95cc3-4f56-4882-bb8b-91b9c111727c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.781963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.782570 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.782879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.783142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.784191 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.786232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4566b-a0c8-4126-8f78-b7d96eeac360 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.798350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1304/2629] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:13.805157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:13.816567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487181b-c28f-4094-a055-54df89e66d22 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:13.822043 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:13.822794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.823066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.823343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] PATH_INFO: `/v3/projects/ce318c1995324b73ad881664d2d27e65/users/9ca0e1ad790241a0a2ce532d89cc142b/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.841308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce318c1995324b73ad881664d2d27e65, user_id=9ca0e1ad790241a0a2ce532d89cc142b, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:13.844015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:13.946068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1326/2630] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:13 2022] PUT /identity/v3/projects/ce318c1995324b73ad881664d2d27e65/users/9ca0e1ad790241a0a2ce532d89cc142b/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:13.957356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:13.957632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:13.957938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:13.996669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] MFA Rules not processed for user `9ca0e1ad790241a0a2ce532d89cc142b`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:13.997911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:14.064244 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1305/2631] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:36:14.068964 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.141475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387995bc-47ac-450a-aff2-dcb8b38a23cc admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.147007 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.147613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.147889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.148150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.148772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.152138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.251810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1327/2632] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:14 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:14.256146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.268320 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86ab47-0569-4e1f-aca2-d91480f21153 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.335049 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.335786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.336142 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.336492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.337211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.339531 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.484955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1306/2633] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:14 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:14.489313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.501358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecc77b-6d33-4608-b36f-8c3c5e3545d7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.506355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.507022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.507296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.507567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.508218 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.509979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.521874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1328/2634] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:14.526764 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.537954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93a214-3a1d-423c-8ace-17a598ad813e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.543915 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.544603 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.544880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.545152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] PATH_INFO: `/v3/projects/a9bea6aafec2415ba884debea545355d/users/61791e56928946a4b8bce582784dee1d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.562293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bea6aafec2415ba884debea545355d, user_id=61791e56928946a4b8bce582784dee1d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.564978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.691052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1307/2635] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:14 2022] PUT /identity/v3/projects/a9bea6aafec2415ba884debea545355d/users/61791e56928946a4b8bce582784dee1d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:14.695140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.709169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e868a-503c-4a4a-90c4-6293b1ae5daa admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.772735 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.773241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.773514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.773815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.774448 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.776250 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6fd00-4c34-4941-97d0-b9663f8bd9b3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.778355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:14.786927 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1329/2636] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:14.792001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.802865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c44cf8-464c-4ce9-a73e-b34168cbee23 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:14.808515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:14.809165 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.809444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.809743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] PATH_INFO: `/v3/projects/a9bea6aafec2415ba884debea545355d/users/61791e56928946a4b8bce582784dee1d/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.811672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9bea6aafec2415ba884debea545355d, user_id=61791e56928946a4b8bce582784dee1d, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:14.814379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:14.825221 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1308/2637] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:14 2022] PUT /identity/v3/projects/a9bea6aafec2415ba884debea545355d/users/61791e56928946a4b8bce582784dee1d/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:14.835854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:14.836338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:14.836553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:14.871105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] MFA Rules not processed for user `61791e56928946a4b8bce582784dee1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:14.871105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:14.944091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1330/2638] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:14.952187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:14.953966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:14.983820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:15.015634 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a216863-1541-485b-9d3a-2f54fbdf8178 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:15.019503 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:15.020104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:15.020384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:15.020657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:15.030412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:15.031831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:15.042312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1309/2639] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:15.173987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:15.175372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:15.187286 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf993-27bd-4f4d-88ec-9f68547774d7 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:15.190886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:15.191473 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:15.191746 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:15.192004 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:15.248086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:15.249544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daa126-ec77-47e7-aa2b-7ed500dd16fc service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:15.258592 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1331/2640] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:20.690785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:20.763163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b0a0-2601-47a0-9003-0297cb88b784 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:20.767769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:20.768396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:20.768671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:20.768942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] PATH_INFO: `/v3/users/9ca0e1ad790241a0a2ce532d89cc142b` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:20.771881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca0e1ad790241a0a2ce532d89cc142b)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:20.772002 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:20.996543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca0e1ad790241a0a2ce532d89cc142b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.001037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca0e1ad790241a0a2ce532d89cc142b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.122528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca0e1ad790241a0a2ce532d89cc142b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.130101 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca0e1ad790241a0a2ce532d89cc142b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.142796 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca0e1ad790241a0a2ce532d89cc142b'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.148934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1310/2641] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:20 2022] DELETE /identity/v3/users/9ca0e1ad790241a0a2ce532d89cc142b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:21.155141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:21.155832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:21.155832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:21.179996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:21.181305 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:21.246466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1332/2642] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:21.256459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:21.257940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:21.310315 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:21.321811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7b5c3-5904-420f-9bd1-990b6dc19e6f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:21.326339 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:21.326957 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:21.327228 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:21.327498 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:21.338111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:21.339555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:21.350119 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1311/2643] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:21.752890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:21.754515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:21.768790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a93a4-32b2-4d52-9683-e2e84dd2b959 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:21.775485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:21.776135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:21.776415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:21.777138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] PATH_INFO: `/v3/projects/ce318c1995324b73ad881664d2d27e65` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:21.778013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce318c1995324b73ad881664d2d27e65)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:21.782153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:21.799235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:21.947282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce318c1995324b73ad881664d2d27e65', 'request_id': 'req-d5a27590-f676-48a0-b2e9-247981474f70'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.953626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce318c1995324b73ad881664d2d27e65', 'request_id': 'req-d5a27590-f676-48a0-b2e9-247981474f70'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:21.996483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce318c1995324b73ad881664d2d27e65', 'request_id': 'req-d5a27590-f676-48a0-b2e9-247981474f70'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.009485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] The token cache is being invalidate because project ce318c1995324b73ad881664d2d27e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:22.009806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.011190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.012800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1333/2644] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:21 2022] DELETE /identity/v3/projects/ce318c1995324b73ad881664d2d27e65 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:22.017733 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:22.027845 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e2a0-fbec-470a-af96-973970880937 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:22.086971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:22.087606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:22.087884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:22.088152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] PATH_INFO: `/v3/users/61791e56928946a4b8bce582784dee1d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:22.089376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61791e56928946a4b8bce582784dee1d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:22.091325 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:22.280960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61791e56928946a4b8bce582784dee1d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.282277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61791e56928946a4b8bce582784dee1d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.356838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61791e56928946a4b8bce582784dee1d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.365268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61791e56928946a4b8bce582784dee1d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.379537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-37a3ec1e-253a-4f27-bfb8-32fcb8e39252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61791e56928946a4b8bce582784dee1d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:22.385654 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1312/2645] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:22 2022] DELETE /identity/v3/users/61791e56928946a4b8bce582784dee1d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:22.840929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:22.840929 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:22.861451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a27590-f676-48a0-b2e9-247981474f70 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:22.921469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:22.922132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:22.922406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:22.922677 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] PATH_INFO: `/v3/projects/a9bea6aafec2415ba884debea545355d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:22.923796 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9bea6aafec2415ba884debea545355d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:22.925698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.053280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9bea6aafec2415ba884debea545355d', 'request_id': 'req-4289eeaa-ff24-429a-9934-8d85a156c51f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:23.061818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9bea6aafec2415ba884debea545355d', 'request_id': 'req-4289eeaa-ff24-429a-9934-8d85a156c51f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:23.133332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9bea6aafec2415ba884debea545355d', 'request_id': 'req-4289eeaa-ff24-429a-9934-8d85a156c51f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:23.144463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] The token cache is being invalidate because project a9bea6aafec2415ba884debea545355d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:23.144754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:23.145406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4289eeaa-ff24-429a-9934-8d85a156c51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:23.146916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1334/2646] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:22 2022] DELETE /identity/v3/projects/a9bea6aafec2415ba884debea545355d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:23.152363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be509cb-347e-430b-a849-77666634ce8b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.152835 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be509cb-347e-430b-a849-77666634ce8b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.152968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be509cb-347e-430b-a849-77666634ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.174111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4be509cb-347e-430b-a849-77666634ce8b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:23.178148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4be509cb-347e-430b-a849-77666634ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:23.248641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1313/2647] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:36:23.256901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.257182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.257517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.281039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:23.282282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:23.290555 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1335/2648] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:23.295762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be509cb-347e-430b-a849-77666634ce8b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.306461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be509cb-347e-430b-a849-77666634ce8b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.311046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.311666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.311942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.312214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.312855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:23.313152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.314787 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.316274 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:23.327467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1314/2649] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:23.340584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.350804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6920-d548-4eba-9d18-2df2cd47c4f8 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.357019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.357628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.357939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.358209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.358837 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.360919 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.486387 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1336/2650] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/projects => generated 389 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:23.490728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.501562 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da20f8-21ff-47cd-a1ee-1a86c4714d04 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.561444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.561998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.562275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.562548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.563196 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.565150 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.672079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1315/2651] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:23.676533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.688072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a590a-3aa2-4043-baf6-3ccb87346b4a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.692742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.693330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.693606 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.693922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.694546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.696186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.697879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:23.707438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1337/2652] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:23.711491 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.721932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8661f0e-c32b-4bd6-9ab8-f40b8cf8bc1c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.727107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.728369 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.729100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.729100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.729324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.730971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea79f32-a861-4a26-85a3-7643f188aac0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.742306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1316/2653] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:23.748001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:23.758312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e035-a794-4b82-8851-c8baa3c7937b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:23.763531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:23.764171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.764444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.764712 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] PATH_INFO: `/v3/projects/d89a7d8aa42b45c790a606351a30a79a/users/bad56fe36a4c4ca5b3840ab564d29806/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.779822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89a7d8aa42b45c790a606351a30a79a, user_id=bad56fe36a4c4ca5b3840ab564d29806, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:23.782683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:23.854987 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1338/2654] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:23 2022] PUT /identity/v3/projects/d89a7d8aa42b45c790a606351a30a79a/users/bad56fe36a4c4ca5b3840ab564d29806/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:36:23.865442 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:23.865738 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:23.866029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:23.895504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] MFA Rules not processed for user `bad56fe36a4c4ca5b3840ab564d29806`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:23.896724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:23.961251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1317/2655] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:23.966495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.028283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a728817-15a7-420a-88d9-56cfb7821528 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.032896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.033775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.034053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.034318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.034945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.036833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.129673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1339/2656] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:23 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:24.134460 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.145525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1b67c-1af6-4021-ad16-15431efd5a19 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.209245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.209977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.210161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.210434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.211079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.214270 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.223661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:24.359510 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1318/2657] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:24 2022] POST /identity/v3/users => generated 302 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:24.365478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.377049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568ba0b-9dea-4310-a27b-84deb850aa8a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.382851 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.383456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.386053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.386053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.386053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.394631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.404903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1340/2658] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:24.410380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.421074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd003575-21d8-42ca-bf67-380eb6125bcc admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.425971 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.426648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.426978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.427324 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] PATH_INFO: `/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc/users/8eda4f2e4c4d4924af2179eae806ceed/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.444419 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe9fd63a50d46618c694b1b4e9221cc, user_id=8eda4f2e4c4d4924af2179eae806ceed, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.447696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.546484 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1319/2659] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:24 2022] PUT /identity/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc/users/8eda4f2e4c4d4924af2179eae806ceed/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:24.551190 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.562029 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4f2a4-aab0-45ed-90e6-f21a90f0c483 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.621066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.621892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.622171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.622447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.623096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.624946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a36a3-8421-44e5-835c-0e227ae8b2b7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.637414 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1341/2660] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:24.641887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.656628 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022a130-b713-4278-ba5d-3792e540620f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.659721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.660476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.660818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.661154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] PATH_INFO: `/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc/users/8eda4f2e4c4d4924af2179eae806ceed/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.664104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe9fd63a50d46618c694b1b4e9221cc, user_id=8eda4f2e4c4d4924af2179eae806ceed, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.667580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.744558 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1320/2661] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:24 2022] PUT /identity/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc/users/8eda4f2e4c4d4924af2179eae806ceed/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:24.744793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.744793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.744793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.746467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] MFA Rules not processed for user `8eda4f2e4c4d4924af2179eae806ceed`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:24.748547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:24.782146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:24.822676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1342/2662] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:24.833308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.835178 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:24.894363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4249-d8bf-460e-8c22-b733d85eeb7d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.898030 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.898736 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.899014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.899385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:24.908832 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:24.910303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:24.919952 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1321/2663] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:24.968139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:24.969619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:24.982673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deb9ee-c449-4f5c-adb4-5c0d800c7441 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:24.986235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:24.986836 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:24.987112 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:24.987383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:25.045810 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:25.047254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23087cbf-450b-4d01-aed4-a636a7e8066c service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:25.056933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1343/2664] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:25.871960 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:25.874708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:25.942548 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28257335-476d-4143-b5c8-0d6abd95ec86 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:25.948184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:25.948824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:25.949123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:25.949396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] PATH_INFO: `/v3/users/a9c9215a987d482983ead148d79c04e4` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:25.950617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c9215a987d482983ead148d79c04e4)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:25.952601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:26.130940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c9215a987d482983ead148d79c04e4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:26.131526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c9215a987d482983ead148d79c04e4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:26.233600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c9215a987d482983ead148d79c04e4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:26.238837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:26.241851 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c9215a987d482983ead148d79c04e4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:26.254303 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c9215a987d482983ead148d79c04e4'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:26.261032 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1322/2665] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:25 2022] DELETE /identity/v3/users/a9c9215a987d482983ead148d79c04e4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:26.265924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:26.266430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:26.266540 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:26.289504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:26.290778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:26.353786 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1344/2666] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:26.363649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:26.365456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:26.431301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ee1c7-c646-4ec8-9b42-0fbe216fdfd3 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:26.435131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:26.435750 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:26.436029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:26.436301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:26.445933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:26.448094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005295d6-b238-4ee5-97f9-9caf2e4646a2 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:26.458184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1323/2667] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:27.276353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:27.278244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:27.292344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41acdb5-7f48-42c4-b1ec-b0d0fad65ce0 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:27.297876 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:27.298708 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.298901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.299178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] PATH_INFO: `/v3/projects/8b4cb0fcf746423d893a463f1826b97b` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.300314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4cb0fcf746423d893a463f1826b97b)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:27.302371 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:27.405576 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4cb0fcf746423d893a463f1826b97b', 'request_id': 'req-1ca7a90a-666d-4446-9325-05df56dce68b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:27.412252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4cb0fcf746423d893a463f1826b97b', 'request_id': 'req-1ca7a90a-666d-4446-9325-05df56dce68b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:27.482106 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4cb0fcf746423d893a463f1826b97b', 'request_id': 'req-1ca7a90a-666d-4446-9325-05df56dce68b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:27.483947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:27.495118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] The token cache is being invalidate because project 8b4cb0fcf746423d893a463f1826b97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:27.495413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:27.496076 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-1ca7a90a-666d-4446-9325-05df56dce68b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:27.497568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1345/2668] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:27 2022] DELETE /identity/v3/projects/8b4cb0fcf746423d893a463f1826b97b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:27.502752 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.503239 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.503364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.523983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:27.525844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:27.586458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1324/2669] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:27.595428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e768e05-1293-425e-9c1b-351274967892 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.601740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e768e05-1293-425e-9c1b-351274967892 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.601740 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e768e05-1293-425e-9c1b-351274967892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.620283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-8e768e05-1293-425e-9c1b-351274967892 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:27.621514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8e768e05-1293-425e-9c1b-351274967892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:27.629569 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1346/2670] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:27.634843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:27.644240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9fa7-f6b5-47dd-8c8e-c46b764b599c None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:27.648860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:27.649511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.650211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.650211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.650843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:27.651140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:27.652789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:27.665675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1325/2671] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:27.670228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e768e05-1293-425e-9c1b-351274967892 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:27.679422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e768e05-1293-425e-9c1b-351274967892 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:27.684993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:27.685584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.685901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.686167 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.686839 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:27.688649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:27.781001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1347/2672] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:27 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:27.785687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:27.796795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ad37-fda1-4d2e-8ccb-39605c044cc4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:27.856626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:27.857240 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:27.857516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:27.857814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:27.858439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:27.860406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:27.872402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:27.980493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1326/2673] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:27 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:27.984736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:27.995792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b1dcf-b32c-40e1-ac1c-7b63fba6972b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:28.000615 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:28.002969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:28.002969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:28.002969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:28.002969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:28.004054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:28.015224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1348/2674] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:28.019170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:28.029824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961eaa6-f83a-4352-9a50-b4d0e18dc13f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:28.034923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:28.037371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:28.037371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:28.037371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:28.037371 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:28.038337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0e5b9-2fd7-4fe2-95a4-0597464d6ef6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:28.051210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1327/2675] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:36:28.054779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:28.065766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79818e2b-9f3f-45ff-b877-44474a18d410 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:28.070654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:28.071507 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:28.071977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:28.072263 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] PATH_INFO: `/v3/projects/f1a5c4aae77c4a5d90f4899b64881e33/users/b6042e2947324660af6438ad7a563c62/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:28.087335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a5c4aae77c4a5d90f4899b64881e33, user_id=b6042e2947324660af6438ad7a563c62, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:28.090091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:28.204399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1349/2676] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:28 2022] PUT /identity/v3/projects/f1a5c4aae77c4a5d90f4899b64881e33/users/b6042e2947324660af6438ad7a563c62/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:28.216696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:28.217267 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:28.217785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:28.261133 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] MFA Rules not processed for user `b6042e2947324660af6438ad7a563c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:28.262273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:28.344307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1328/2677] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:28.356209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:28.358222 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:28.418517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8332-380e-43f3-ab6c-b6d202c43c3b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:28.422185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:28.422854 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:28.423121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:28.423385 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:28.433136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:28.434853 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f3b94-2d95-444e-8cb6-1147449fbc17 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:28.444389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1350/2678] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:31.224657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:31.226659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:31.308499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dd960-3271-4512-ab54-79c942124d88 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:31.314355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:31.314992 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:31.315265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:31.315533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] PATH_INFO: `/v3/users/bad56fe36a4c4ca5b3840ab564d29806` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:31.316671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad56fe36a4c4ca5b3840ab564d29806)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:31.318646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:31.595148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad56fe36a4c4ca5b3840ab564d29806'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:31.595148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad56fe36a4c4ca5b3840ab564d29806'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:31.784982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad56fe36a4c4ca5b3840ab564d29806'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:31.797388 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad56fe36a4c4ca5b3840ab564d29806'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:31.809126 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:31.811534 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad56fe36a4c4ca5b3840ab564d29806'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:31.817656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1329/2679] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:31 2022] DELETE /identity/v3/users/bad56fe36a4c4ca5b3840ab564d29806 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:36:31.823099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:31.823979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:31.824259 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:31.846367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:31.847621 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:31.918354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1351/2680] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:31.931569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:31.931569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:31.999962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89afc3-a26a-4117-a288-6534ec402396 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:32.003546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:32.004159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:32.004435 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:32.004703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:32.015382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:32.016831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:32.027999 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1330/2681] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:32.552428 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:32.554037 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:32.569552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44f90a-fc6c-4d61-92c2-a95125c5a769 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:32.575640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:32.576131 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:32.576525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:32.576687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] PATH_INFO: `/v3/projects/d89a7d8aa42b45c790a606351a30a79a` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:32.577967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89a7d8aa42b45c790a606351a30a79a)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:32.579877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:32.607614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:32.734664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89a7d8aa42b45c790a606351a30a79a', 'request_id': 'req-5a3f440e-9155-4273-90f7-71631adc1be3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:32.741008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89a7d8aa42b45c790a606351a30a79a', 'request_id': 'req-5a3f440e-9155-4273-90f7-71631adc1be3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:32.853936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89a7d8aa42b45c790a606351a30a79a', 'request_id': 'req-5a3f440e-9155-4273-90f7-71631adc1be3'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:32.866620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] The token cache is being invalidate because project d89a7d8aa42b45c790a606351a30a79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:32.866907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:32.867559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:32.869757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1352/2682] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:32 2022] DELETE /identity/v3/projects/d89a7d8aa42b45c790a606351a30a79a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:32.876715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:32.888166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f4e46-fadb-474d-8e94-8bf96f78a8f1 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:32.946780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:32.947399 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:32.947673 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:32.947941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] PATH_INFO: `/v3/users/8eda4f2e4c4d4924af2179eae806ceed` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:32.949075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eda4f2e4c4d4924af2179eae806ceed)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:32.951003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:33.159704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eda4f2e4c4d4924af2179eae806ceed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.160309 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eda4f2e4c4d4924af2179eae806ceed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.247428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eda4f2e4c4d4924af2179eae806ceed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.254935 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eda4f2e4c4d4924af2179eae806ceed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.269241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-2f63db4b-5b7a-4bd1-804a-3bb91856b461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eda4f2e4c4d4924af2179eae806ceed'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.275944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1331/2683] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:32 2022] DELETE /identity/v3/users/8eda4f2e4c4d4924af2179eae806ceed => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:33.733289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:33.734798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:33.748484 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f440e-9155-4273-90f7-71631adc1be3 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:33.807069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:33.807700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:33.807982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:33.808318 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] PATH_INFO: `/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:33.809466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe9fd63a50d46618c694b1b4e9221cc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:33.811422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:33.928055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe9fd63a50d46618c694b1b4e9221cc', 'request_id': 'req-b3639679-c790-470d-beb3-ee0ee231d163'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.935485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe9fd63a50d46618c694b1b4e9221cc', 'request_id': 'req-b3639679-c790-470d-beb3-ee0ee231d163'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:33.990262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe9fd63a50d46618c694b1b4e9221cc', 'request_id': 'req-b3639679-c790-470d-beb3-ee0ee231d163'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:34.002290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] The token cache is being invalidate because project bfe9fd63a50d46618c694b1b4e9221cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:34.002583 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:34.003217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b3639679-c790-470d-beb3-ee0ee231d163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:34.004714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1353/2684] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:33 2022] DELETE /identity/v3/projects/bfe9fd63a50d46618c694b1b4e9221cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:34.010430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.010943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.011063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.031040 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:34.032281 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:34.042602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:34.095716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1332/2685] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:34.105396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.112695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.112695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.130944 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:34.132264 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:34.140506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1354/2686] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:36:34.145041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.155961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e6e17-1334-467d-bb59-270b70815233 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.162053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.162806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.163070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.163286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.163942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:34.164241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.166688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.185986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1333/2687] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:34.190315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.200145 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f4021-636b-401a-b255-22b2c2b1704f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.205008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.205614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.205959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.206235 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.206861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.208975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.318879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1355/2688] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:34.323289 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.333685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a5b0f-4ffd-436b-91a8-685a37eeac79 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.400897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.401556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.401831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.402127 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.402934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.404909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.505459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1334/2689] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:34.516085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.536968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ad594-864f-423e-99e5-8af4dd9c3c00 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.543603 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.545238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.545475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.546733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.546733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.549504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.562488 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1356/2690] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:34.568356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.580029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a554-38e9-4929-abe9-5eaafba6c633 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.587635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.589881 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.590342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.590759 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.591620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.594184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11a770-9bb7-4f29-bca8-287f2e127ea4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.620184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1335/2691] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:34.624696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.630023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:34.636561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7f4d-c0b4-488c-b874-fdcf6e5fdbe1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.642060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.642731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.643010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.643356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] PATH_INFO: `/v3/projects/d796a16b177b493fab06b6c2f3c45d18/users/35d1f09fad5c4db396ec40e0b2a9482d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.666236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d796a16b177b493fab06b6c2f3c45d18, user_id=35d1f09fad5c4db396ec40e0b2a9482d, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.668979 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:34.739931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1357/2692] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:34 2022] PUT /identity/v3/projects/d796a16b177b493fab06b6c2f3c45d18/users/35d1f09fad5c4db396ec40e0b2a9482d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:34.751050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.751410 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.751690 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.782770 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] MFA Rules not processed for user `35d1f09fad5c4db396ec40e0b2a9482d`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:34.784311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:34.857685 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1336/2693] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:34.862283 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:34.927920 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d66c95-e6bc-44d6-9779-62980430f0a9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:34.932723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:34.933421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:34.933643 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:34.933937 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:34.934560 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:34.936514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.061778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1358/2694] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:34 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:36:35.066202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.078536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406d42-a2f3-4046-89dc-4f099856f396 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.087139 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:35.138046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.138577 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.138857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.139131 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.139762 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.141683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.232257 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1337/2695] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:35 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:35.238303 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.249114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414d0fe-7caf-4147-8a2b-194ee621e230 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.254719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.255326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.255600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.255866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.256510 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.258135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.270930 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1359/2696] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:35.275372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.286718 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd2d6b-8b72-4b03-a7b5-e0cae3813b5e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.292923 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.293284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.293440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.293681 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] PATH_INFO: `/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0/users/57c571b378c94597831d44da6e37967b/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.310708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8bffb6f6f3f4059844c1a78f972c5a0, user_id=57c571b378c94597831d44da6e37967b, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.314452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.391669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1338/2697] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:35 2022] PUT /identity/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0/users/57c571b378c94597831d44da6e37967b/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:35.395846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.406999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb17e6-3e92-4195-87fb-805483606860 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.466543 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.467160 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.467438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.467711 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.468350 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.469970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c27abc-c755-4c34-bac0-a729319f5379 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.480901 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1360/2698] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:35.485444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.496452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046b42b-2f39-48f0-be04-88e3b546fe05 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.504780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.504780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.504780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.504780 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] PATH_INFO: `/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0/users/57c571b378c94597831d44da6e37967b/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.505175 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8bffb6f6f3f4059844c1a78f972c5a0, user_id=57c571b378c94597831d44da6e37967b, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.507160 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.517928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1339/2699] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:35 2022] PUT /identity/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0/users/57c571b378c94597831d44da6e37967b/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:35.529430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.529829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.530172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.558744 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] MFA Rules not processed for user `57c571b378c94597831d44da6e37967b`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:35.559963 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:35.601632 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:35.631124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1361/2700] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:35.642458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:35.644152 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:35.702703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f7015-a3cc-49ea-8eab-57a3729e3cda admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:35.706454 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:35.707054 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:35.707331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:35.707601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:35.716843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:35.718362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd6623-1a86-4615-b5ca-1e8868a7d86c service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:35.728326 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1340/2701] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:37.085285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:37.166085 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ae64-c9ce-4e2a-a315-2793435bbf6e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:37.170955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:37.171669 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:37.172018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:37.172357 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] PATH_INFO: `/v3/users/35d1f09fad5c4db396ec40e0b2a9482d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:37.173571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d1f09fad5c4db396ec40e0b2a9482d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:37.175648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:37.382402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d1f09fad5c4db396ec40e0b2a9482d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:37.383071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d1f09fad5c4db396ec40e0b2a9482d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:37.391420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d1f09fad5c4db396ec40e0b2a9482d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:37.510771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d1f09fad5c4db396ec40e0b2a9482d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:37.511396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d1f09fad5c4db396ec40e0b2a9482d'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:37.534114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1362/2702] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:37 2022] DELETE /identity/v3/users/35d1f09fad5c4db396ec40e0b2a9482d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:36:37.539200 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:37.539687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:37.539801 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:37.560499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:37.561986 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:37.628375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1341/2703] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:37.639128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:37.640550 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:37.703083 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a17c3-5cd3-400a-9537-41538d1e0e00 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:37.706763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:37.707370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:37.707646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:37.707917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:37.715882 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:37.718532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:37.719976 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:37.741968 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1363/2704] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:38.186907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:38.189120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:38.204940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f2e85-801a-4027-b8ae-20aacd54f3e2 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:38.210363 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:38.211917 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:38.212357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:38.212788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] PATH_INFO: `/v3/projects/d796a16b177b493fab06b6c2f3c45d18` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:38.215249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d796a16b177b493fab06b6c2f3c45d18)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:38.217233 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:38.219346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:38.374543 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd796a16b177b493fab06b6c2f3c45d18', 'request_id': 'req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.381060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd796a16b177b493fab06b6c2f3c45d18', 'request_id': 'req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.464515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd796a16b177b493fab06b6c2f3c45d18', 'request_id': 'req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.476854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] The token cache is being invalidate because project d796a16b177b493fab06b6c2f3c45d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:38.477921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.478520 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.479978 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1342/2705] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:38 2022] DELETE /identity/v3/projects/d796a16b177b493fab06b6c2f3c45d18 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:38.484135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:38.494475 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fb335-5b18-4977-bdba-5b58d4135d55 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:38.556123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:38.557018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:38.557301 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:38.557577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] PATH_INFO: `/v3/users/57c571b378c94597831d44da6e37967b` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:38.558775 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c571b378c94597831d44da6e37967b)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:38.561153 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:38.782229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c571b378c94597831d44da6e37967b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.782889 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c571b378c94597831d44da6e37967b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.790413 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c571b378c94597831d44da6e37967b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.864658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c571b378c94597831d44da6e37967b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.865282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f3571490-b6cf-43a3-a588-45a13028ca2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c571b378c94597831d44da6e37967b'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:38.884545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1364/2706] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:38 2022] DELETE /identity/v3/users/57c571b378c94597831d44da6e37967b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:39.440418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:39.442277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:39.456035 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d693e4-3d08-4c3c-a9b3-c40daa2cd4cf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:39.526900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:39.527532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:39.527810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:39.528078 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] PATH_INFO: `/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:39.529214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8bffb6f6f3f4059844c1a78f972c5a0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:39.531265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:39.695662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8bffb6f6f3f4059844c1a78f972c5a0', 'request_id': 'req-560eccf4-9521-4227-9ec0-00f3cf70e473'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:39.702578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8bffb6f6f3f4059844c1a78f972c5a0', 'request_id': 'req-560eccf4-9521-4227-9ec0-00f3cf70e473'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:39.779125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8bffb6f6f3f4059844c1a78f972c5a0', 'request_id': 'req-560eccf4-9521-4227-9ec0-00f3cf70e473'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:39.792379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] The token cache is being invalidate because project a8bffb6f6f3f4059844c1a78f972c5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:39.792677 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:39.793346 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-560eccf4-9521-4227-9ec0-00f3cf70e473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:39.794936 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1343/2707] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:39 2022] DELETE /identity/v3/projects/a8bffb6f6f3f4059844c1a78f972c5a0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:39.800268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:39.801082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:39.801082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:39.822285 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:39.823733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:39.885186 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1365/2708] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:36:39.893616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:39.893927 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:39.894199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:39.915546 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:39.918052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:39.925563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1344/2709] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:39.929814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:39.939959 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4e684-4380-4811-b934-80da6a6c74d2 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:39.944466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:39.945062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:39.945345 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:39.945597 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:39.946397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:39.946616 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:39.948193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:39.960474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1366/2710] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:39.966149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:39.975452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e98f0d-4314-415f-a7cc-3b3bb2678981 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:39.981073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:39.981664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:39.981974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:39.982246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:39.982884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:39.984864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:39.996234 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:40.196838 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1345/2711] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:39 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:40.199496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:40.205687 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:40.211185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda42ffe-6253-4be3-b83b-f0fef1a4c1c5 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:40.272427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:40.273039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.273320 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.273590 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.274829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:40.292861 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:40.482284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1367/2712] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:40 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:40.487670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:40.497831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15a865-9e94-4f8a-8a0b-82f788ab6603 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:40.502825 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:40.503423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.503704 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.503964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.504602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:40.506210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:40.517514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1346/2713] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:40.525223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:40.538038 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e0ba5-abec-4704-8aed-9b6c859ce645 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:40.543009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:40.554770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.554770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.554770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.554770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:40.554770 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16376a1a-5200-4326-90fb-1b5ae73b4a25 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:40.562446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1368/2714] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:36:40.563637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:40.574277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dbaf1-cb03-48bf-a24d-a09f01f3d527 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:40.578912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:40.579559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.579836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.580108 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] PATH_INFO: `/v3/projects/5686c0ae742a4f8ca92b4c9e0045d0c3/users/01c0ada9306c4746be35ed8a84dd7c5e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.595083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5686c0ae742a4f8ca92b4c9e0045d0c3, user_id=01c0ada9306c4746be35ed8a84dd7c5e, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:40.597753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:40.729151 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1347/2715] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:40 2022] PUT /identity/v3/projects/5686c0ae742a4f8ca92b4c9e0045d0c3/users/01c0ada9306c4746be35ed8a84dd7c5e/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:36:40.740720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.740945 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.741206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.770719 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] MFA Rules not processed for user `01c0ada9306c4746be35ed8a84dd7c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:40.771934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:40.840331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1369/2716] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:40.852705 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:40.854687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:40.913019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b5a74-860e-41e1-8403-5ec2d548cc34 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:40.917383 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:40.918027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:40.918308 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:40.918580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:40.927232 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:40.928659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff73217-76eb-4c98-a493-c7f34fc875df service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:40.938626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1348/2717] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:42.992509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:42.994608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:43.060214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c3266-33e8-403b-9010-b48641e07639 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:43.064726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:43.065347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:43.065622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:43.065916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] PATH_INFO: `/v3/users/b6042e2947324660af6438ad7a563c62` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:43.067018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6042e2947324660af6438ad7a563c62)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:43.068908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:43.081173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:43.399778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6042e2947324660af6438ad7a563c62'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:43.400459 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6042e2947324660af6438ad7a563c62'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:43.407389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6042e2947324660af6438ad7a563c62'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:43.532794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6042e2947324660af6438ad7a563c62'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:43.533437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6042e2947324660af6438ad7a563c62'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:43.552904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1370/2718] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:42 2022] DELETE /identity/v3/users/b6042e2947324660af6438ad7a563c62 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:43.557659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:43.558159 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:43.558286 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:43.577406 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:43.578693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:43.639288 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1349/2719] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:43.650247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:43.651596 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:43.709313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43b34-c8bb-4e03-bba8-3f257c6c4d70 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:43.714095 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:43.718802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:43.718802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:43.718802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:43.724690 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:43.726180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ff4e-d607-45e0-8019-b54cfe9d0006 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:43.736905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1371/2720] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:44.169745 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:44.171265 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:44.187344 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a2095-fc5d-4b70-a549-3d979587db16 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:44.190385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:44.191036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.191307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.191569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] PATH_INFO: `/v3/projects/f1a5c4aae77c4a5d90f4899b64881e33` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.192794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a5c4aae77c4a5d90f4899b64881e33)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:44.194760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:44.374767 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a5c4aae77c4a5d90f4899b64881e33', 'request_id': 'req-fda907c9-9238-48b6-9c6c-bee7c2b44feb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:44.380966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a5c4aae77c4a5d90f4899b64881e33', 'request_id': 'req-fda907c9-9238-48b6-9c6c-bee7c2b44feb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:44.382781 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:44.504316 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a5c4aae77c4a5d90f4899b64881e33', 'request_id': 'req-fda907c9-9238-48b6-9c6c-bee7c2b44feb'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:44.515629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] The token cache is being invalidate because project f1a5c4aae77c4a5d90f4899b64881e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:44.516067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:44.516716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fda907c9-9238-48b6-9c6c-bee7c2b44feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:44.518242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1350/2721] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:44 2022] DELETE /identity/v3/projects/f1a5c4aae77c4a5d90f4899b64881e33 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:44.523258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.523768 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.523953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.545769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:44.546985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:44.606866 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1372/2722] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:44.614619 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.618223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.618223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.636636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:44.637933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:44.645941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1351/2723] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:44.649941 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:44.659832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918d65b-6668-4739-88cf-8eaf8a3d7c2f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:44.664983 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:44.665538 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.665855 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.666179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.666769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:44.667067 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:44.668654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:44.680931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1373/2724] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:44.685296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:44.694559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3b6e6-d54a-4a37-8156-a16eb3b635a0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:44.699379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:44.699963 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.700230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.700493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.701111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:44.703156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:44.803171 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1352/2725] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:44 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:44.808450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:44.818580 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856b00-1d71-4d49-8ac3-c14481a17d0b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:44.835307 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:44.876196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:44.876797 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:44.877078 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:44.877351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:44.878017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:44.880118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.008197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1374/2726] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:44 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:36:45.012602 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:45.023301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a01e-2086-47d9-baab-bf30e1eb4dce admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:45.029201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:45.029830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.030156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.034312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.034312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:45.034312 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.044208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1353/2727] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:45.048420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:45.060591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae5c8d-935d-45de-9979-04da3f741b94 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:45.065322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:45.065954 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.066258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.066530 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.067198 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:45.068777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53d9dc-ba4d-48f3-8d7a-befc2f4781e1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.079818 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1375/2728] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:45.084073 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:45.095568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f5415-0863-4e3c-8b05-c7a9ea4bb022 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:45.100079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:45.100715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.100987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.101252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] PATH_INFO: `/v3/projects/856da1c4c94042f1b0a7c0cc6d7e2765/users/b6bd603b76c243189004932d4434f912/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.116387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] RBAC: Authorizing `identity:create_grant(project_id=856da1c4c94042f1b0a7c0cc6d7e2765, user_id=b6bd603b76c243189004932d4434f912, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:45.119230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.190964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1354/2729] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:45 2022] PUT /identity/v3/projects/856da1c4c94042f1b0a7c0cc6d7e2765/users/b6bd603b76c243189004932d4434f912/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:45.201189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.201469 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.201769 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.231471 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] MFA Rules not processed for user `b6bd603b76c243189004932d4434f912`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:45.232782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:45.300831 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1376/2730] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:45.311954 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:45.313739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:45.371245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867b673-7cbf-4d78-8439-829bf8f9d31a admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:45.375015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:45.375610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.375888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.376154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.385043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:45.386490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a611-4e8c-4e90-813d-0b821af39a51 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.395166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1355/2731] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:45.504999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:45.506412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:45.578092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9d920-5609-46b7-86f7-737934f39636 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:45.581118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:45.581792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.582074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.582343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] PATH_INFO: `/v3/users/01c0ada9306c4746be35ed8a84dd7c5e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.583468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c0ada9306c4746be35ed8a84dd7c5e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:45.585411 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:45.819460 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:45.833458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c0ada9306c4746be35ed8a84dd7c5e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:45.834175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c0ada9306c4746be35ed8a84dd7c5e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:45.841505 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c0ada9306c4746be35ed8a84dd7c5e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:45.932619 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c0ada9306c4746be35ed8a84dd7c5e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:45.933229 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c0ada9306c4746be35ed8a84dd7c5e'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:45.953426 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1377/2732] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:36:45 2022] DELETE /identity/v3/users/01c0ada9306c4746be35ed8a84dd7c5e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:45.958430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:45.958430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:45.958953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:45.979739 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:45.980939 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:46.041774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1356/2733] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:36:46.051802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:46.053532 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:46.111905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ae40-13f9-4749-a5ed-2e635955b23f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:46.115377 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:46.115971 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:46.116243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:46.116506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:46.126458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:46.127879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03143249-87b1-491c-aafc-5758f5f35014 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:46.138297 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1378/2734] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:46.631799 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:46.633283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:46.646003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f1e81-8b49-499b-a1fc-3fa650e60afe None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:46.651730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:46.652364 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:46.652646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:46.652916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] PATH_INFO: `/v3/projects/5686c0ae742a4f8ca92b4c9e0045d0c3` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:46.654741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5686c0ae742a4f8ca92b4c9e0045d0c3)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:46.656928 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:46.676006 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:46.825826 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5686c0ae742a4f8ca92b4c9e0045d0c3', 'request_id': 'req-8db18abd-96f1-491b-a005-97872d96ca22'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:46.831897 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5686c0ae742a4f8ca92b4c9e0045d0c3', 'request_id': 'req-8db18abd-96f1-491b-a005-97872d96ca22'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:47.030688 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5686c0ae742a4f8ca92b4c9e0045d0c3', 'request_id': 'req-8db18abd-96f1-491b-a005-97872d96ca22'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:47.042517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] The token cache is being invalidate because project 5686c0ae742a4f8ca92b4c9e0045d0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:36:47.042805 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:47.043464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-8db18abd-96f1-491b-a005-97872d96ca22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:36:47.045010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1357/2735] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:36:46 2022] DELETE /identity/v3/projects/5686c0ae742a4f8ca92b4c9e0045d0c3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:47.049880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.050375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.050483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.070011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:47.071239 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:47.132416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1379/2736] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:47.197636 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.198109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.198254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.218089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:47.219368 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:47.227163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1358/2737] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:47.231728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.241561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288930c-eb98-4330-ad86-f01a1ae33225 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.246813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.247420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.247695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.247961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.248617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:36:47.248906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.250548 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:47.265290 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1380/2738] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:36:47.284890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.294700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b239f2-95e9-46cc-9a17-b20ba74a55b0 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.304463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.305070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.305402 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.306413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.307343 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.309661 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:47.510476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1359/2739] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:36:47 2022] POST /identity/v3/projects => generated 371 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:47.515389 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.526737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4f8a-f6f7-46d4-a0a8-79a16ffaad7d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.563447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:47.586089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.586676 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.586960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.587313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.587969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.590265 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:47.751487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1381/2740] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:36:47 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:36:47.755855 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.766146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1470de-d806-4ede-bcb9-e089bd82f3f6 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.770984 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.771580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.771913 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.772183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.772840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.774472 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:47.785310 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1360/2741] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:36:47.789420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.800252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d388454-3f6e-44bd-8033-86a2113f9908 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.804907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.805537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.805966 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.806790 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.807446 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.809062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88a781-6d29-4d6c-830d-4b9790700a66 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:47.821843 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1382/2742] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:36:47.825007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:47.835649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e47cc-f2b9-42e6-b2b3-02b60dffa98f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:47.840737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:47.841381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:47.841655 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:47.841959 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] PATH_INFO: `/v3/projects/a1ec50e36c0b42e6b065c51489502e62/users/3229169c85c24b0895c63049ab7b29aa/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:47.857293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ec50e36c0b42e6b065c51489502e62, user_id=3229169c85c24b0895c63049ab7b29aa, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:47.859944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:48.002106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1361/2743] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:36:47 2022] PUT /identity/v3/projects/a1ec50e36c0b42e6b065c51489502e62/users/3229169c85c24b0895c63049ab7b29aa/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:36:48.012600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:48.012883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:48.013159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:48.043269 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] MFA Rules not processed for user `3229169c85c24b0895c63049ab7b29aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:36:48.044523 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:48.108447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1383/2744] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:36:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:36:48.120097 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:36:48.121911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:36:48.201808 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf2fc-4019-44bc-8484-2d9043e80b34 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:36:48.205533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:36:48.206170 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:36:48.206453 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:36:48.206726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:36:48.216092 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:36:48.217526 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14025afb-a798-4783-b4b1-e9d1d7b41c14 service glance] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:36:48.226777 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1362/2745] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:36:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:36:58.105652 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:36:58.222517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:15.098969 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:15.100402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:15.165760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14706b3f-1421-4b1c-8d8f-425c3f0d50a7 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:15.170152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:15.170776 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:15.171053 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:15.171322 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] PATH_INFO: `/v3/users/b6bd603b76c243189004932d4434f912` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:15.172432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6bd603b76c243189004932d4434f912)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:15.174383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:15.364333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6bd603b76c243189004932d4434f912'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:15.364990 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6bd603b76c243189004932d4434f912'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:15.372047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6bd603b76c243189004932d4434f912'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:15.452440 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6bd603b76c243189004932d4434f912'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:15.453018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6bd603b76c243189004932d4434f912'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:15.472777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1384/2746] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:37:15 2022] DELETE /identity/v3/users/b6bd603b76c243189004932d4434f912 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:15.477521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:15.478053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:15.478167 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:15.497249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:15.498485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:15.560794 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1363/2747] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:15.571238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:15.572754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:15.637772 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682729fe-897e-4ca6-ba72-c2149d7cb2c2 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:15.641633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:15.642281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:15.642559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:15.642832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:15.652203 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:15.653631 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024c830-219b-48cc-8323-9ccc7db3e6c3 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:15.664397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1385/2748] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:16.111359 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.112819 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:16.125408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14bc5a-bac7-4896-bf04-a9312d088432 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.131005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.131645 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.131922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.132188 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] PATH_INFO: `/v3/projects/856da1c4c94042f1b0a7c0cc6d7e2765` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.133335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=856da1c4c94042f1b0a7c0cc6d7e2765)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.135284 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.298768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856da1c4c94042f1b0a7c0cc6d7e2765', 'request_id': 'req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:16.305180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856da1c4c94042f1b0a7c0cc6d7e2765', 'request_id': 'req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:16.406988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856da1c4c94042f1b0a7c0cc6d7e2765', 'request_id': 'req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:16.418317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] The token cache is being invalidate because project 856da1c4c94042f1b0a7c0cc6d7e2765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:37:16.418989 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:16.419293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4279ab3e-b519-4c58-ab3e-6cf27a7bd466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:16.420814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1364/2749] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:37:16 2022] DELETE /identity/v3/projects/856da1c4c94042f1b0a7c0cc6d7e2765 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:16.425896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.426396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.426500 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.446736 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:16.447957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:16.465700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:16.507893 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1386/2750] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:16.515888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.516370 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.516572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.537282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:16.538523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:16.546940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1365/2751] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:16.552019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.561403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e10291-5c1c-4fdb-8554-d82e3b36f47f None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.566830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.567451 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.567722 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.567991 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.568668 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:37:16.568960 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.570566 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.582522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1387/2752] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:37:16.587452 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.596983 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f58ee4-2cf5-4f91-b7c9-209a8db66449 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.601667 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.602290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.602568 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.602836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.603469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.608973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.711070 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1366/2753] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:37:16 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:37:16.716048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.726665 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86bc3b9-3f50-4919-b1b7-e8758a4fd779 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.788036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.788635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.788912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.789176 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.789848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.791728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.911380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1388/2754] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:37:16 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:37:16.915563 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.926907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec92e65-7383-41bf-aac5-48010dd39b6b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.931606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.932212 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.932485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.932753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.933393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.935020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.948776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1367/2755] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:37:16.950109 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:16.962496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a988c-1a8c-4206-9702-37c933407dbf admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:16.967123 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:16.967738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:16.968013 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:16.968282 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:16.968924 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:16.970551 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e21a8-2a6a-422b-8a0a-138d628a5afe admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:16.983138 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1389/2756] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:37:16.989981 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:17.000342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a75b3-55cc-43aa-9613-6ece18b51b26 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:17.005698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:17.006385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:17.006663 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:17.006938 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] PATH_INFO: `/v3/projects/634952274d2748b988c4610754edc371/users/43fefdc28eb54c9f8401bf11b8935065/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:17.023058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=634952274d2748b988c4610754edc371, user_id=43fefdc28eb54c9f8401bf11b8935065, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:17.025741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:17.092691 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1368/2757] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:37:16 2022] PUT /identity/v3/projects/634952274d2748b988c4610754edc371/users/43fefdc28eb54c9f8401bf11b8935065/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:17.098407 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29144-9a87-4b04-af25-d4a305bddd13 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:17.098692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29144-9a87-4b04-af25-d4a305bddd13 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:17.098965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29144-9a87-4b04-af25-d4a305bddd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:17.118931 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-dce29144-9a87-4b04-af25-d4a305bddd13 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:17.120230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dce29144-9a87-4b04-af25-d4a305bddd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:17.181478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1390/2758] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:17.191450 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:17.193173 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:17.215176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:17.250374 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e3ba4-dddf-4a59-ba84-f5f79a305d4f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:17.257507 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:17.258167 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:17.258444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:17.258707 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:17.269039 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:17.270517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:17.280513 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1369/2759] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:23.114938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.115766 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.116045 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.159237 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] MFA Rules not processed for user `43fefdc28eb54c9f8401bf11b8935065`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:23.161783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:23.188199 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:23.244972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1391/2760] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:23.248514 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.259584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbdc50-b35e-48d3-b16a-786e57098c3d service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.264436 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.265132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.265625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.266009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.266716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.268720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:23.409010 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1370/2761] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:37:23 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:37:23.415201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.426896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419bb69-3ef9-4a80-9fb1-ec016122e01d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.516142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.516601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.516880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.517151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.517862 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.520119 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:23.631240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1392/2762] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:37:23 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:37:23.637675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.649578 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f1a4c-80a2-4a02-9e0a-9275cd8d0245 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.654757 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.664879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.664879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.664879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.664879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.664879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:23.671041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1371/2763] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:37:23.674955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.686427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edeee0-c420-4ee6-9311-b62f6a35acb5 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.699490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.699490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.699490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.699490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] PATH_INFO: `/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df/users/f52b32d40cde4c48ab20cd9e50d21fd5/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.710993 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2f03184ce48ffa7b6b3f8c7cf75df, user_id=f52b32d40cde4c48ab20cd9e50d21fd5, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.713684 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:23.830341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1393/2764] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:37:23 2022] PUT /identity/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df/users/f52b32d40cde4c48ab20cd9e50d21fd5/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:23.838023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.854606 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c971c0e-ee85-4949-bcd2-fccaaa61e980 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.946753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.947396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.947722 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.947996 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.948662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.950319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd91813-c0d7-418b-9f32-1ce915709cc2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:23.962387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1372/2765] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:37:23.969086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:23.978382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4db71-fb66-4394-b87a-47c3bd2ad28a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:23.984071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:23.984761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:23.985040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:23.985309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] PATH_INFO: `/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df/users/f52b32d40cde4c48ab20cd9e50d21fd5/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:23.987423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea2f03184ce48ffa7b6b3f8c7cf75df, user_id=f52b32d40cde4c48ab20cd9e50d21fd5, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:23.990185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:24.002478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1394/2766] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:37:23 2022] PUT /identity/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df/users/f52b32d40cde4c48ab20cd9e50d21fd5/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:30.283025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9653-e62b-431f-8aac-98b2e6ce9f30 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:30.283569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9653-e62b-431f-8aac-98b2e6ce9f30 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:30.284018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b9653-e62b-431f-8aac-98b2e6ce9f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:30.316949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0c1b9653-e62b-431f-8aac-98b2e6ce9f30 None None] MFA Rules not processed for user `f52b32d40cde4c48ab20cd9e50d21fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:30.318317 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0c1b9653-e62b-431f-8aac-98b2e6ce9f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:30.395076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1373/2767] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:37:30.409171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:30.411080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:30.481287 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da68f70-f9dd-4e42-bf6b-29ab180dd977 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:30.485511 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be3a2d2b02f44ce9bd64ffd38675b6d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:30.486174 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:30.486454 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:30.486725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:30.514412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:30.549556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:30.551546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:30.561817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1395/2768] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:31.008398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:31.013044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:31.013044 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:31.030467 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:31.065389 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:31.067412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:31.143345 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1374/2769] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:37:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:31.297625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:31.305693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:31.418450 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078055ad-eeaa-44b9-a6b5-7ac9f8de368c service glance] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:31.423795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:31.424417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:31.424695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:31.424970 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:31.434985 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:31.436638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441c2d4-753d-44e3-adef-c203a73d220c service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:31.446915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1396/2770] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:37:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 21:37:37.554026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:37.554026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:37.679404 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e41909-547a-43e6-b04f-b9cde82de9eb None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:37.685027 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:37.685697 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:37.686009 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:37.686285 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] PATH_INFO: `/v3/users/3229169c85c24b0895c63049ab7b29aa` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:37.687540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3229169c85c24b0895c63049ab7b29aa)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:37.689624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:38.461199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3229169c85c24b0895c63049ab7b29aa'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:38.461882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3229169c85c24b0895c63049ab7b29aa'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:38.560869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3229169c85c24b0895c63049ab7b29aa'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:38.570601 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3229169c85c24b0895c63049ab7b29aa'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:38.583708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3229169c85c24b0895c63049ab7b29aa'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:38.595028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1375/2771] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:37:37 2022] DELETE /identity/v3/users/3229169c85c24b0895c63049ab7b29aa => generated 0 bytes in 1042 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:38.596528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:38.597035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:38.597245 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:38.622354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:38.624006 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:38.703705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1397/2772] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:37:38.713965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:38.715461 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:38.723392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:38.786445 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb98a1-29e9-44be-bc40-3adc254bb639 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:38.790627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:38.795176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:38.795176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:38.795176 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:38.802613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:38.804385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc157d6-7553-4dee-8e36-d5b8ddc1e1ba service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:38.816462 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1376/2773] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:39.337294 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:39.340846 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:39.368683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15543-c81a-43b5-afbc-92e606c5223d None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:39.380825 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:39.382048 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:39.382595 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:39.383260 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] PATH_INFO: `/v3/projects/a1ec50e36c0b42e6b065c51489502e62` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:39.386084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ec50e36c0b42e6b065c51489502e62)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:39.393437 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:39.541170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:39.561181 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ec50e36c0b42e6b065c51489502e62', 'request_id': 'req-612446f0-d292-4c93-b03c-d3d94c372c74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:39.568758 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ec50e36c0b42e6b065c51489502e62', 'request_id': 'req-612446f0-d292-4c93-b03c-d3d94c372c74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:39.669178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ec50e36c0b42e6b065c51489502e62', 'request_id': 'req-612446f0-d292-4c93-b03c-d3d94c372c74'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:39.684072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] The token cache is being invalidate because project a1ec50e36c0b42e6b065c51489502e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:37:39.684624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:39.685332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:37:39.697880 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1398/2774] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:37:39 2022] DELETE /identity/v3/projects/a1ec50e36c0b42e6b065c51489502e62 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:37:43.149465 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:43.150085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:43.150085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:43.171586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] MFA Rules not processed for user `43fefdc28eb54c9f8401bf11b8935065`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:43.172833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:43.231674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1377/2775] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:37:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:48.688124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:48.689468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:48.771036 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612446f0-d292-4c93-b03c-d3d94c372c74 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:48.777080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:48.777728 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:48.778022 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:48.778296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:48.780243 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:48.875068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:48.876680 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:48.898040 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1399/2776] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:49.385765 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:49.387208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:49.452824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f40c61-0516-4ad1-bb06-1d8de12d28ab None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:49.456408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:49.457017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:49.457293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:49.458257 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:49.460887 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:49.533074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:49.536990 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b839a-dcb0-454c-b58b-ce57a42342ff service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:49.549490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1378/2777] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:49.940142 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:49.941644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:49.973071 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:50.013432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c212a21-f476-438e-a63a-17adf29abf50 service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:50.018683 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:50.019366 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:50.019646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:50.019916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:50.022942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:50.038403 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:50.040128 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:50.052982 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1400/2778] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:37:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:37:54.710114 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f013-0f3f-4be5-8b5e-81d213ebe3fb None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:54.710653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f013-0f3f-4be5-8b5e-81d213ebe3fb None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:54.710785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36f013-0f3f-4be5-8b5e-81d213ebe3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:54.719341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=100 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:37:54.731713 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-4a36f013-0f3f-4be5-8b5e-81d213ebe3fb None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:37:54.732933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4a36f013-0f3f-4be5-8b5e-81d213ebe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:54.791457 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1379/2779] 10.222.0.99 () {58 vars in 1009 bytes} [Fri Sep 2 21:37:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:37:54.807780 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:37:54.813777 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:37:54.896833 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1cf459-4a70-4048-b6ba-246033df267c service placement] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:37:54.902463 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:37:54.903089 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:37:54.903720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:37:54.903720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:37:54.914707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:37:54.920726 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:37:54.927662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1401/2780] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:37:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:38:02.716515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:02.716515 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:02.718003 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:02.737846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] MFA Rules not processed for user `e9450f3f7527406ab70fcf0b64cab787`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:02.739169 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:02.747004 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1380/2781] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:04.923696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=52 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:12.732094 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=26 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:33.621912 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:33.623355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:33.682956 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7a889-9ac4-468c-b0e3-4fb44d15e192 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:33.686617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:33.687217 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:33.687489 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:33.688903 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:33.690699 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:33.765182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:33.766664 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:33.777787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1402/2782] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:35.072046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:35.073481 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:35.085755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6c452-650c-43cf-8a53-74ba1d9ccd2a None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:35.089647 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:35.090262 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:35.090729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:35.090997 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:35.092724 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:35.176768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:35.178847 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:35.189944 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1381/2783] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:35.319891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:35.329240 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8b690-52a7-434d-919a-85ff519a6d6f service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:35.333639 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:35.335220 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:35.335794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:35.335794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:35.337852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:35.406674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:35.408140 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:35.418826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1403/2784] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:39.365811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:39.368390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:39.402069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b24a-861f-4422-88d7-190e21b151d0 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:39.408778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:39.410521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:39.410833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:39.411109 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] PATH_INFO: `/v3/users/7d2ee46e8302468bb90ee913eeeba4af` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:39.413059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2ee46e8302468bb90ee913eeeba4af)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:39.415198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:39.648028 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2ee46e8302468bb90ee913eeeba4af'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:39.653237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2ee46e8302468bb90ee913eeeba4af'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:39.742737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2ee46e8302468bb90ee913eeeba4af'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:39.753456 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2ee46e8302468bb90ee913eeeba4af'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:39.791026 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-13ec068b-67a7-48f9-8220-4d36f1c12d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2ee46e8302468bb90ee913eeeba4af'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:39.807504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1382/2785] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:38:39 2022] DELETE /identity/v3/users/7d2ee46e8302468bb90ee913eeeba4af => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:40.562228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:40.584073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d374c-5cbd-4769-b883-f4ce3a44b668 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:40.679228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:40.713223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:40.716612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:40.716612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:40.716612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] PATH_INFO: `/v3/projects/de8efd74da71400a83f05127451c13fd` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:40.716612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8efd74da71400a83f05127451c13fd)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:40.718867 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:40.877434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8efd74da71400a83f05127451c13fd', 'request_id': 'req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:40.886624 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8efd74da71400a83f05127451c13fd', 'request_id': 'req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:40.954252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8efd74da71400a83f05127451c13fd', 'request_id': 'req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:40.974173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] The token cache is being invalidate because project de8efd74da71400a83f05127451c13fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:38:40.974173 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:40.975646 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-18e8b95f-bfa9-45ff-a0d0-eb1bee316377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:40.979445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1404/2786] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:38:40 2022] DELETE /identity/v3/projects/de8efd74da71400a83f05127451c13fd => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:40.987791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:40.987791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:40.987791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.028788 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:41.031080 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:41.152528 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1383/2787] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:38:41.160258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.160531 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.160803 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.183276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:41.184504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:41.192840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1405/2788] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:41.197141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.206976 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36084ba6-836d-4205-84b6-2acdc450db33 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.212516 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.213331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.213706 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.214120 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.214876 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:38:41.215296 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.217095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.219034 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:41.237224 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1384/2789] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 21:38:41.238519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.249028 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6dbd-3162-40b7-9635-9dc5de06159e None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.260122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.260805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.261102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.261382 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.262068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.264477 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.362212 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1406/2790] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:38:41 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:41.366922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.377955 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6205b2-9d2d-4b4f-a609-d5ff58ee7a2c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.441158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.442586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.442586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.442586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.443213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.446210 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.548827 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1385/2791] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:38:41 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:38:41.553622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.564465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d1cc1-aeba-49d9-8fe9-7d74f3bfc721 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.570039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.570697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.570932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.571195 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.571840 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.573444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.590019 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1407/2792] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:41.591412 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.603925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065288ee-4e94-4bb4-b38e-46fbb4f339cf admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.621820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.621820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.621820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.621820 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.622504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.624211 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb749b51-a081-4652-9d47-024aaef6dd4d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.653319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1386/2793] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:41.658380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.671238 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872141bf-9194-438f-b6ef-433af0dd1857 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.676165 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:41.676921 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.677308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.677649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] PATH_INFO: `/v3/projects/fd54f33f56184ebc83c0bfd1514b2bfa/users/9111c4f319ec4414a8b55a6d45c6ec71/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.696072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd54f33f56184ebc83c0bfd1514b2bfa, user_id=9111c4f319ec4414a8b55a6d45c6ec71, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:41.698972 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:41.794274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1408/2794] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:41 2022] PUT /identity/v3/projects/fd54f33f56184ebc83c0bfd1514b2bfa/users/9111c4f319ec4414a8b55a6d45c6ec71/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:41.805773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:41.806213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:41.806572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:41.837213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] MFA Rules not processed for user `9111c4f319ec4414a8b55a6d45c6ec71`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:41.838458 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:41.902932 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1387/2795] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:41.907886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:41.994361 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6150156-5e9d-4865-ae85-c87a5acfef6b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:41.999585 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.000192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.000468 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.000793 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.001434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.003788 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.020425 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:42.114601 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1409/2796] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:38:41 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:42.118384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.129605 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494b66d-d0ce-4685-b907-294a79fabd77 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.194749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.195586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.195924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.196198 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.197023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.199106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.208437 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:42.313380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1388/2797] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:38:42 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:38:42.317358 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.329567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b989bc0-52a3-40ab-8eef-ff95de1eddce admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.336328 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.336947 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.337225 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.337518 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.338223 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.340105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.352495 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1410/2798] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:42.356716 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.368387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4d10-b570-42cb-98fe-9cda6273e066 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.372911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.373564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.373916 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.374187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] PATH_INFO: `/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55/users/4577133cb384450eaf0e37a0c2ca5c18/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.390106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9526f0ce14a41ff94c8c900cc9a0a55, user_id=4577133cb384450eaf0e37a0c2ca5c18, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.392807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.462883 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1389/2799] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:42 2022] PUT /identity/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55/users/4577133cb384450eaf0e37a0c2ca5c18/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:42.467953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.477957 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417c140-175d-4188-8d2f-41b95af2b6d7 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.535753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.536299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.536577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.536848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.537556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.539236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ed668-c661-4d90-b3bc-a139e5b4ab40 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.549476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1411/2800] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:38:42.554524 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.564709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98484887-b078-4d41-b15f-a71bf0aa4cb4 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.570005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.575574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.575574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.575574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] PATH_INFO: `/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55/users/4577133cb384450eaf0e37a0c2ca5c18/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.575574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9526f0ce14a41ff94c8c900cc9a0a55, user_id=4577133cb384450eaf0e37a0c2ca5c18, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.575968 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.586509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1390/2801] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:42 2022] PUT /identity/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55/users/4577133cb384450eaf0e37a0c2ca5c18/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:42.597524 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.598494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.599228 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.629276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] MFA Rules not processed for user `4577133cb384450eaf0e37a0c2ca5c18`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:42.630498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:42.700211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1412/2802] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:38:42.713420 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:42.715043 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:42.782597 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308dee64-868e-436e-972d-f84c432c2f0e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:42.787814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:42.788802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:42.789086 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:42.789361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:42.799356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:42.800865 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827244a-5bdb-4a54-82bb-c4e10552632f service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:42.815882 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1391/2803] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:38:44.052262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:44.145081 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586826-4d30-4cff-9ddf-4113feb1f179 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:44.150415 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:44.151135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:44.151467 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:44.154965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] PATH_INFO: `/v3/users/9111c4f319ec4414a8b55a6d45c6ec71` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:44.154965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9111c4f319ec4414a8b55a6d45c6ec71)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:44.154965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:44.161187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:44.472242 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9111c4f319ec4414a8b55a6d45c6ec71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:44.472942 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9111c4f319ec4414a8b55a6d45c6ec71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:44.481943 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9111c4f319ec4414a8b55a6d45c6ec71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:44.569648 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9111c4f319ec4414a8b55a6d45c6ec71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:44.570314 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9111c4f319ec4414a8b55a6d45c6ec71'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:44.589379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1413/2804] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:38:44 2022] DELETE /identity/v3/users/9111c4f319ec4414a8b55a6d45c6ec71 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:44.594059 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750958-0758-4ed6-9821-625458fb0095 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:44.594687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750958-0758-4ed6-9821-625458fb0095 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:44.594912 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50750958-0758-4ed6-9821-625458fb0095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:44.616384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50750958-0758-4ed6-9821-625458fb0095 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:44.617613 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50750958-0758-4ed6-9821-625458fb0095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:44.680482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1392/2805] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:44.691761 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:44.693832 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:44.769304 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fe1a0-6c3e-4ab9-b26a-7b26aa3bdf3b admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:44.773933 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:44.774552 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:44.775175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:44.775175 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:44.785231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:44.786695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:44.797438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1414/2806] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:45.304637 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50750958-0758-4ed6-9821-625458fb0095 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:45.306161 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50750958-0758-4ed6-9821-625458fb0095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:45.319642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50750958-0758-4ed6-9821-625458fb0095 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:45.323973 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:45.324610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:45.324886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:45.325154 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] PATH_INFO: `/v3/projects/fd54f33f56184ebc83c0bfd1514b2bfa` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:45.326331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd54f33f56184ebc83c0bfd1514b2bfa)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:45.328263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:45.468866 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd54f33f56184ebc83c0bfd1514b2bfa', 'request_id': 'req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:45.470771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:45.475609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd54f33f56184ebc83c0bfd1514b2bfa', 'request_id': 'req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:45.571052 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd54f33f56184ebc83c0bfd1514b2bfa', 'request_id': 'req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:45.581828 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] The token cache is being invalidate because project fd54f33f56184ebc83c0bfd1514b2bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:38:45.582408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:45.583554 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:45.585925 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1393/2807] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:38:45 2022] DELETE /identity/v3/projects/fd54f33f56184ebc83c0bfd1514b2bfa => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:38:45.589914 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:45.600073 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fe5a8-0d1e-4e11-ac4a-949295c62b0c service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:45.661003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:45.661644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:45.662472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:45.662749 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] PATH_INFO: `/v3/users/4577133cb384450eaf0e37a0c2ca5c18` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:45.663887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4577133cb384450eaf0e37a0c2ca5c18)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:45.665932 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:45.988789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:46.009008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4577133cb384450eaf0e37a0c2ca5c18'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.009008 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4577133cb384450eaf0e37a0c2ca5c18'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.010946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4577133cb384450eaf0e37a0c2ca5c18'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.164895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4577133cb384450eaf0e37a0c2ca5c18'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.165517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4ba02f85-174d-4238-aafe-5c59b58cf2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4577133cb384450eaf0e37a0c2ca5c18'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.184546 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1415/2808] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:38:45 2022] DELETE /identity/v3/users/4577133cb384450eaf0e37a0c2ca5c18 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:46.744980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:46.746702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:46.761907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615c5e0-12bf-4b5b-ac77-dffc54f1ccc0 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:46.823106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:46.823743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:46.824018 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:46.824290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] PATH_INFO: `/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:46.825413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9526f0ce14a41ff94c8c900cc9a0a55)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:46.827381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:46.955795 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9526f0ce14a41ff94c8c900cc9a0a55', 'request_id': 'req-32706d1e-0b5b-45fd-9ee2-bc750178b313'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:46.962495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9526f0ce14a41ff94c8c900cc9a0a55', 'request_id': 'req-32706d1e-0b5b-45fd-9ee2-bc750178b313'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:47.095811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9526f0ce14a41ff94c8c900cc9a0a55', 'request_id': 'req-32706d1e-0b5b-45fd-9ee2-bc750178b313'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:47.110069 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] The token cache is being invalidate because project d9526f0ce14a41ff94c8c900cc9a0a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:38:47.110610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:47.111189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-32706d1e-0b5b-45fd-9ee2-bc750178b313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:38:47.112768 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1394/2809] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:38:46 2022] DELETE /identity/v3/projects/d9526f0ce14a41ff94c8c900cc9a0a55 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:47.118694 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.119151 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.119278 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.139575 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:47.140789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:47.203321 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1416/2810] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:47.211340 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.211617 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.211886 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.232087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:47.233294 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:47.242072 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1395/2811] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:47.246397 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.256571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8145258-e7ce-4a10-b871-f44741830e6b None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.263529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.264066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.264343 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.264612 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.265258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81842) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:38:47.265559 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.267197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.279258 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1417/2812] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:38:47.283669 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.293693 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2e3d-5d52-49bd-b97e-bab0bffb5b76 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.298620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.299214 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.299485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.299749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.300379 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.302298 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.413029 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1396/2813] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:47.417329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.428617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03005814-1f40-4cd8-998a-cb36c183edb8 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.487710 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.488329 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.488611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.488886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.489525 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.491759 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.614105 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1418/2814] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:47.618589 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.629942 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab67d-7aec-47a5-a21a-3d80694c22f7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.635074 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.635694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.635972 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.636248 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.636899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.638586 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.650269 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1397/2815] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:47.654509 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.666581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662c028-da09-4a28-a8a2-73abe4a0870d admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.671271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.671897 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.672169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.672441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.673084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.674707 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b592bd-f38d-41e5-90a1-b43bd402fe37 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.686115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1419/2816] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:47.690407 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.700761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f11f85-ace5-419d-96a5-857775cdfb9c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.705413 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.706125 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.706408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.706682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] PATH_INFO: `/v3/projects/ffacd71bc7fb44ee9b9912b57bef167d/users/71c3a56b74604e9c96aea2a83f3f8f06/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.722230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffacd71bc7fb44ee9b9912b57bef167d, user_id=71c3a56b74604e9c96aea2a83f3f8f06, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.724964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:47.805867 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1398/2817] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:47 2022] PUT /identity/v3/projects/ffacd71bc7fb44ee9b9912b57bef167d/users/71c3a56b74604e9c96aea2a83f3f8f06/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:47.816783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.817064 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.817337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.844433 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:47.847432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] MFA Rules not processed for user `71c3a56b74604e9c96aea2a83f3f8f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:47.848654 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:47.916136 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1420/2818] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:47.920662 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:47.929860 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:47.982593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42591b4-a09a-45b7-b832-fa735fb0f719 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:47.988020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:47.988621 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:47.988900 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:47.989181 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:47.989853 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:47.992180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.080490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1399/2819] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:38:47 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:48.084591 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.096311 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b478bd9-681a-4a47-a28c-a44c23b4b296 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.156267 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.157034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.157187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.157453 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] PATH_INFO: `/v3/users` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.158108 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.160515 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.284376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1421/2820] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:38:48 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:38:48.286334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.297400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8f8c5-f156-4f4a-a4c1-8270640be1b8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.305744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.306192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.306466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.306734 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.307393 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.308966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.321036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1400/2821] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:38:48.325436 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.335599 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fbc77-1883-470a-936a-e92dbce2f182 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.340356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.341021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.341327 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.341581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] PATH_INFO: `/v3/projects/248f854795c24c129c14707423de9458/users/f5b8f1b4e0ae43f49e515cbf1a23ea03/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.361900 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=248f854795c24c129c14707423de9458, user_id=f5b8f1b4e0ae43f49e515cbf1a23ea03, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.364635 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.514938 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1422/2822] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:48 2022] PUT /identity/v3/projects/248f854795c24c129c14707423de9458/users/f5b8f1b4e0ae43f49e515cbf1a23ea03/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:48.519149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.529253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e491c-9928-46ae-9182-d78bb87fe268 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.591806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.592425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.592708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.592987 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.593683 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.595305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed292ce4-ea88-444b-b829-6ee82795068e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.608342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1401/2823] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:38:48.610442 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.621849 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c006b-2685-4457-9269-e32029900ceb admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.626506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.627157 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.627434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.627714 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] PATH_INFO: `/v3/projects/248f854795c24c129c14707423de9458/users/f5b8f1b4e0ae43f49e515cbf1a23ea03/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.629627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=248f854795c24c129c14707423de9458, user_id=f5b8f1b4e0ae43f49e515cbf1a23ea03, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.632334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.642844 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1423/2824] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:38:48 2022] PUT /identity/v3/projects/248f854795c24c129c14707423de9458/users/f5b8f1b4e0ae43f49e515cbf1a23ea03/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:38:48.654266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.654687 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.654961 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.685110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] MFA Rules not processed for user `f5b8f1b4e0ae43f49e515cbf1a23ea03`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:38:48.686366 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:48.758418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1402/2825] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:38:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:38:48.769869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:48.771673 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:38:48.833333 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515612e-7730-4635-9743-440dff9131f0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:48.837537 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:48.838211 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:48.838519 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:48.838778 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:48.847905 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:48.849337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:48.854016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:48.860010 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1424/2826] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:49.121665 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:38:49.126273 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:49.132751 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b41776-aee6-4346-a962-fd4a4eaf9cdc None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:38:49.136441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:38:49.137038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:38:49.137311 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:38:49.137579 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:38:49.195351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:38:49.196773 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:38:49.207085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1403/2827] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:38:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:38:59.027393 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:38:59.203975 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=27 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:01.612640 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:01.614196 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:01.674479 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae851b7-3169-48b7-ae79-8488556d6f79 service cinder] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:01.678179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:01.678821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:01.679100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:01.679375 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:01.681801 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:01.692874 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:01.694423 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5682af5-3f8c-4a0d-8470-c26faad9a757 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:01.703952 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1425/2828] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:02.784347 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:02.852189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d17fa2-a1fa-4b22-97d5-c3bac8be5939 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:02.858063 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:02.858702 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:02.858974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:02.859237 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] PATH_INFO: `/v3/users/71c3a56b74604e9c96aea2a83f3f8f06` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:02.860386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c3a56b74604e9c96aea2a83f3f8f06)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:02.862387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:03.095107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c3a56b74604e9c96aea2a83f3f8f06'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:03.095843 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c3a56b74604e9c96aea2a83f3f8f06'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:03.325017 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c3a56b74604e9c96aea2a83f3f8f06'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:03.339148 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c3a56b74604e9c96aea2a83f3f8f06'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:03.354892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c3a56b74604e9c96aea2a83f3f8f06'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:03.364482 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1404/2829] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:02 2022] DELETE /identity/v3/users/71c3a56b74604e9c96aea2a83f3f8f06 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:03.366316 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:03.366812 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:03.366906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:03.388641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:03.389888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:03.455216 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1426/2830] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:03.465879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:03.467443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:03.537045 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f25f39-6113-4c71-bb3f-962dd198b533 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:03.541384 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:03.542023 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:03.542392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:03.543892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:03.556110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:03.558219 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:03.568275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1405/2831] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:04.007091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:04.008545 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:04.022197 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8f65f-db1a-44d9-a326-f006c3b87061 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:04.027528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:04.028169 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:04.028452 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:04.028725 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] PATH_INFO: `/v3/projects/ffacd71bc7fb44ee9b9912b57bef167d` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:04.029884 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffacd71bc7fb44ee9b9912b57bef167d)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:04.032121 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:04.293184 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffacd71bc7fb44ee9b9912b57bef167d', 'request_id': 'req-b05ddd38-1b2b-40a5-8d78-4673e278f073'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.306755 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffacd71bc7fb44ee9b9912b57bef167d', 'request_id': 'req-b05ddd38-1b2b-40a5-8d78-4673e278f073'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.370868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffacd71bc7fb44ee9b9912b57bef167d', 'request_id': 'req-b05ddd38-1b2b-40a5-8d78-4673e278f073'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.383061 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] The token cache is being invalidate because project ffacd71bc7fb44ee9b9912b57bef167d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:04.383364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.383986 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.385483 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1427/2832] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:04 2022] DELETE /identity/v3/projects/ffacd71bc7fb44ee9b9912b57bef167d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:04.389858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:04.399541 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac59a9-ba34-4a2d-927d-c31fadc88c82 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:04.419527 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:04.458357 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:04.458922 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:04.459203 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:04.459475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] PATH_INFO: `/v3/users/f5b8f1b4e0ae43f49e515cbf1a23ea03` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:04.460600 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b8f1b4e0ae43f49e515cbf1a23ea03)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:04.462895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:04.699183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b8f1b4e0ae43f49e515cbf1a23ea03'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.699891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b8f1b4e0ae43f49e515cbf1a23ea03'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.782335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b8f1b4e0ae43f49e515cbf1a23ea03'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.792110 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b8f1b4e0ae43f49e515cbf1a23ea03'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.804581 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-428fde03-3b0c-4ce9-ae25-9d46a628fb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b8f1b4e0ae43f49e515cbf1a23ea03'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:04.811335 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1406/2833] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:04 2022] DELETE /identity/v3/users/f5b8f1b4e0ae43f49e515cbf1a23ea03 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:39:05.288034 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:05.291214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:05.310150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ddd38-1b2b-40a5-8d78-4673e278f073 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:05.352908 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:05.408491 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:05.409899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:05.410396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:05.410869 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] PATH_INFO: `/v3/projects/248f854795c24c129c14707423de9458` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:05.412697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=248f854795c24c129c14707423de9458)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:05.417256 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:05.587720 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248f854795c24c129c14707423de9458', 'request_id': 'req-5b618507-2844-4526-bb25-01adf7aff1f2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:05.594416 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248f854795c24c129c14707423de9458', 'request_id': 'req-5b618507-2844-4526-bb25-01adf7aff1f2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:06.166356 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248f854795c24c129c14707423de9458', 'request_id': 'req-5b618507-2844-4526-bb25-01adf7aff1f2'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:06.179421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] The token cache is being invalidate because project 248f854795c24c129c14707423de9458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:06.182039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:06.182039 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-5b618507-2844-4526-bb25-01adf7aff1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:06.185787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1428/2834] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:05 2022] DELETE /identity/v3/projects/248f854795c24c129c14707423de9458 => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:06.187868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.188386 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.188486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.212396 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:06.213657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:06.301334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1407/2835] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:39:06.310021 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.310310 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.310584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.332857 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:06.334206 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:06.343529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1429/2836] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:06.349532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.366355 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7101624-15e6-46f6-9b9b-55739e5e299b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.372155 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.372791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.373067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.373338 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.374038 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:39:06.374495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.376138 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.390079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1408/2837] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:39:06.394814 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.406886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168f2c2-b358-4dd3-9e46-0c2e93c223ab None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.411967 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.412662 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.412841 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.413107 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.413799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.416625 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.572610 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1430/2838] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:06 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:39:06.576962 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.587253 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b6315-64d9-4686-9651-0097404661cb admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.614382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:06.646473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.647005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.647282 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.647552 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.648180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.650260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.765943 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1409/2839] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:06 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:39:06.770281 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.781187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5c3d-5c30-4157-90d8-c40bb5a727e9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.786396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.797688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.797688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.797688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.797688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.797688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.801533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1431/2840] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:06.805684 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.817500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781c8e7-a210-4f9b-b085-56a64ef2295b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.822337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.822945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.823249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.823521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.824163 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.825771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e3fa9-bce3-455f-9722-01c4401b3754 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.836060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1410/2841] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:06.840464 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:06.850678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c4a5-76f7-4808-af3b-794b2b04f344 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:06.856003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:06.856644 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:06.856917 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:06.857193 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] PATH_INFO: `/v3/projects/67e95bed5fd447fea09fe5b07512925e/users/4a3e35673f534c9e8edb1a06276c5592/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:06.873041 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e95bed5fd447fea09fe5b07512925e, user_id=4a3e35673f534c9e8edb1a06276c5592, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:06.875787 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:06.994087 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1432/2842] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:06 2022] PUT /identity/v3/projects/67e95bed5fd447fea09fe5b07512925e/users/4a3e35673f534c9e8edb1a06276c5592/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:07.004675 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.005042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.005387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.037653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] MFA Rules not processed for user `4a3e35673f534c9e8edb1a06276c5592`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:07.038924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:07.111350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1411/2843] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:39:07.116024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.181126 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:07.194703 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4918a8-3e7b-47a1-8e8c-1a7f26ecea56 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.199925 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.200544 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.200820 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.201094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.201748 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.203729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.300904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1433/2844] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:07.305517 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.318112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27b94f-5fcf-425f-a4a5-f9596f1b2d43 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.379958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.380641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.380920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.381180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.381846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.384387 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.488998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1412/2845] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:07.494068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.504574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3af1df-3a8a-4552-9a29-e7a042a76bd1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.509458 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.510103 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.510383 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.510653 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.511296 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.517813 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.523794 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1434/2846] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:07.527896 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.538397 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b853b4c-fe4e-4abf-a7b7-5bbba5811f75 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.543931 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.544540 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.544846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.545112 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.545858 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.547502 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e648fac9-f140-41ec-a976-a5e0b04192e3 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.558489 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1413/2847] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:07.562689 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.573372 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67b061-fd8a-47be-9e07-f0d85e5cfbfc admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.578012 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.578659 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.578935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.579204 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] PATH_INFO: `/v3/projects/f2d39cfea4294ce38130fffc1f0ad117/users/4436d2e0252e428ea0ab16954ac2bf02/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.594444 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d39cfea4294ce38130fffc1f0ad117, user_id=4436d2e0252e428ea0ab16954ac2bf02, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.597086 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.677852 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1435/2848] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:07 2022] PUT /identity/v3/projects/f2d39cfea4294ce38130fffc1f0ad117/users/4436d2e0252e428ea0ab16954ac2bf02/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:07.689229 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.689532 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.689844 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.718506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] MFA Rules not processed for user `4436d2e0252e428ea0ab16954ac2bf02`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:07.719725 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:07.723893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:07.784441 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1414/2849] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:07.788992 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.852834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545bef5-2ce8-4b08-b23d-fe80d43b5de0 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:07.857564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:07.858189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:07.858465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:07.858729 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:07.859346 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:07.861230 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:07.938331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1436/2850] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:07.942251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:07.953459 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf6d08-4d7f-4adb-9bec-da6c29994e11 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.009216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.009700 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.009998 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.010259 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.010892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.012790 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.106351 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1415/2851] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:07 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:08.110490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.121159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c7c1-e70a-42a0-bfc3-a324b144770e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.125935 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.126529 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.126804 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.127069 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.127695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.129268 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.140753 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1437/2852] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:08.145087 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.156055 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f319fc-98d0-4ed5-80da-925b7c775bf7 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.160970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.161629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.161949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.162215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] PATH_INFO: `/v3/projects/1a42a70e383f40648c1599fd6f658d71/users/96c5c7ccb0b943e69cb9e967d243c740/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.177111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a42a70e383f40648c1599fd6f658d71, user_id=96c5c7ccb0b943e69cb9e967d243c740, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.179786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.262672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1416/2853] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:08 2022] PUT /identity/v3/projects/1a42a70e383f40648c1599fd6f658d71/users/96c5c7ccb0b943e69cb9e967d243c740/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:39:08.266737 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.277951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0ffcc-7a3f-4678-9748-95c9aaeac71e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.350779 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.351402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.351688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.352139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.353835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.355481 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de5be3-2374-4983-a067-59a819075545 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.371881 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1438/2854] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:08.376075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.387755 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047f289-4360-41d0-b62a-4ae9c989bb9b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.392525 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.393949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.393949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.393949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] PATH_INFO: `/v3/projects/1a42a70e383f40648c1599fd6f658d71/users/96c5c7ccb0b943e69cb9e967d243c740/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.395760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a42a70e383f40648c1599fd6f658d71, user_id=96c5c7ccb0b943e69cb9e967d243c740, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.398463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.411400 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1417/2855] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:08 2022] PUT /identity/v3/projects/1a42a70e383f40648c1599fd6f658d71/users/96c5c7ccb0b943e69cb9e967d243c740/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:08.422798 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.423277 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.423564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.456410 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:08.460447 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] MFA Rules not processed for user `96c5c7ccb0b943e69cb9e967d243c740`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:08.462359 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:08.547506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1439/2856] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:08.559050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.560779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:08.626041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c14b6-e0a2-4d2d-98f6-e54ac80a5a70 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.629574 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.630215 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.630492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.630761 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.641537 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.643014 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.648559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:08.653651 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1418/2857] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:08.770517 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:08.772494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:08.786731 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eabc06-2371-4833-8bed-de1d3ed38bee None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:08.791226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:08.791997 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:08.792834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:08.792834 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:08.857568 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:08.859033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e07a8-178b-40b3-a314-0948fae2910f service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:08.870514 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1440/2858] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:14.198924 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:14.262197 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969257fb-2e22-44b2-acbd-6941fed73414 service cinder] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:14.266899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:14.267533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:14.267793 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:14.268067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] PATH_INFO: `/v3/users/4a3e35673f534c9e8edb1a06276c5592` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:14.269187 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3e35673f534c9e8edb1a06276c5592)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:14.271263 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:14.470050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3e35673f534c9e8edb1a06276c5592'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:14.470719 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3e35673f534c9e8edb1a06276c5592'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:14.547192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3e35673f534c9e8edb1a06276c5592'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:14.555909 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3e35673f534c9e8edb1a06276c5592'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:14.571020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3e35673f534c9e8edb1a06276c5592'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:14.577319 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1419/2859] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:14 2022] DELETE /identity/v3/users/4a3e35673f534c9e8edb1a06276c5592 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:14.582637 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:14.583164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:14.583164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:14.604506 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:14.609533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:14.668907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1441/2860] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:14.678021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:14.679440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:14.739627 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15341e-97d5-4a2d-896c-00b58ee9056e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:14.743812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:14.744477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:14.744830 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:14.745158 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:14.754480 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:14.755940 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:14.766076 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1420/2861] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:15.230367 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:15.232003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:15.246115 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b164a-4a93-4bff-8995-6a9af3a9c682 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:15.250891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:15.252904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:15.252904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:15.252904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] PATH_INFO: `/v3/projects/67e95bed5fd447fea09fe5b07512925e` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:15.253308 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e95bed5fd447fea09fe5b07512925e)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:15.255214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:15.394724 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:15.409421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e95bed5fd447fea09fe5b07512925e', 'request_id': 'req-f98a4223-ca0a-4604-95f3-e6c7b78370a9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.416696 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e95bed5fd447fea09fe5b07512925e', 'request_id': 'req-f98a4223-ca0a-4604-95f3-e6c7b78370a9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.522950 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e95bed5fd447fea09fe5b07512925e', 'request_id': 'req-f98a4223-ca0a-4604-95f3-e6c7b78370a9'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.536288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] The token cache is being invalidate because project 67e95bed5fd447fea09fe5b07512925e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:15.536627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.537276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.538946 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1442/2862] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:15 2022] DELETE /identity/v3/projects/67e95bed5fd447fea09fe5b07512925e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:15.543475 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:15.553425 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e7015-e53f-4682-8f05-1e2d6c7a007e service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:15.576521 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:15.616888 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:15.617555 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:15.617953 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:15.618649 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] PATH_INFO: `/v3/users/4436d2e0252e428ea0ab16954ac2bf02` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:15.619492 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4436d2e0252e428ea0ab16954ac2bf02)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:15.621438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:15.890352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4436d2e0252e428ea0ab16954ac2bf02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.891380 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4436d2e0252e428ea0ab16954ac2bf02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:15.996749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4436d2e0252e428ea0ab16954ac2bf02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.004297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4436d2e0252e428ea0ab16954ac2bf02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.016547 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4436d2e0252e428ea0ab16954ac2bf02'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.022753 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1421/2863] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:15 2022] DELETE /identity/v3/users/4436d2e0252e428ea0ab16954ac2bf02 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:16.485306 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:16.486402 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:16.501182 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4223-ca0a-4604-95f3-e6c7b78370a9 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:16.561420 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:16.562091 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:16.562369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:16.562641 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] PATH_INFO: `/v3/projects/f2d39cfea4294ce38130fffc1f0ad117` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:16.563765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d39cfea4294ce38130fffc1f0ad117)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:16.565774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:16.690817 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d39cfea4294ce38130fffc1f0ad117', 'request_id': 'req-06552717-b4fb-4d9d-890c-ffe1c9918ad1'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.697244 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d39cfea4294ce38130fffc1f0ad117', 'request_id': 'req-06552717-b4fb-4d9d-890c-ffe1c9918ad1'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.770674 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d39cfea4294ce38130fffc1f0ad117', 'request_id': 'req-06552717-b4fb-4d9d-890c-ffe1c9918ad1'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.783309 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] The token cache is being invalidate because project f2d39cfea4294ce38130fffc1f0ad117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:16.788549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.788549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:16.788549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1443/2864] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:16 2022] DELETE /identity/v3/projects/f2d39cfea4294ce38130fffc1f0ad117 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:16.791703 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:16.793123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:16.806111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee05c-fa7f-42b1-a4ff-6e19f0268217 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:16.865067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:16.865672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:16.865977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:16.866249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] PATH_INFO: `/v3/users/96c5c7ccb0b943e69cb9e967d243c740` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:16.867401 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c5c7ccb0b943e69cb9e967d243c740)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:16.869509 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:17.047771 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:17.050660 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c5c7ccb0b943e69cb9e967d243c740'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:17.051439 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c5c7ccb0b943e69cb9e967d243c740'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:17.198776 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c5c7ccb0b943e69cb9e967d243c740'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:17.207082 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c5c7ccb0b943e69cb9e967d243c740'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:17.226709 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-96ef19b6-aa5c-48f5-bcf5-5d79cd121b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c5c7ccb0b943e69cb9e967d243c740'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:17.233842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1422/2865] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:16 2022] DELETE /identity/v3/users/96c5c7ccb0b943e69cb9e967d243c740 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:17.813805 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:17.815270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:17.829094 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06552717-b4fb-4d9d-890c-ffe1c9918ad1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:17.887009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:17.896754 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:17.897432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:17.897730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:17.898009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] PATH_INFO: `/v3/projects/1a42a70e383f40648c1599fd6f658d71` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:17.899172 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a42a70e383f40648c1599fd6f658d71)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:17.901178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.054114 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a42a70e383f40648c1599fd6f658d71', 'request_id': 'req-4f59b97b-21dc-4f5d-85df-304306420d9c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:18.060977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a42a70e383f40648c1599fd6f658d71', 'request_id': 'req-4f59b97b-21dc-4f5d-85df-304306420d9c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:18.168200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a42a70e383f40648c1599fd6f658d71', 'request_id': 'req-4f59b97b-21dc-4f5d-85df-304306420d9c'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:18.179723 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] The token cache is being invalidate because project 1a42a70e383f40648c1599fd6f658d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:18.180009 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:18.180655 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-4f59b97b-21dc-4f5d-85df-304306420d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:18.182364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1444/2866] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:17 2022] DELETE /identity/v3/projects/1a42a70e383f40648c1599fd6f658d71 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:18.188630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.189523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.189523 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.209964 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:18.211201 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:18.276630 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1423/2867] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 21:39:18.287799 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.288279 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.288556 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.310146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:18.311364 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:18.319528 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1445/2868] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:18.324128 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.334252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b448-30e4-4954-9f12-0908da3a606b None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.339695 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.340292 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.340572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.340846 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.341494 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:39:18.341837 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.343473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.356012 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1424/2869] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:39:18.360622 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.371192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cfdad-5bd6-4d32-8ef7-1d1cb01434d5 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.376394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.377014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.377284 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.377577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.378241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.380150 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.498354 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1446/2870] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:18 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 21:39:18.502656 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.513580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8db5d7-b410-4ab4-825a-b18215eb7c1b admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.572743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.573699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.573699 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.573970 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.574584 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.576500 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.668358 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1425/2871] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:18 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:18.672716 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.683895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134502c-df54-4fbf-a08f-5a0758738fbf admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.690593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.690593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.690593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.690593 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.691099 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.692110 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.703686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1447/2872] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:39:18.707977 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.719495 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c807a89-2e63-4d9c-acd1-c7f65956d37d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.724490 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.725111 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.725392 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.725672 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.726362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.728061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9b98a-9dac-4389-8117-4714468f5c84 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.739473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1426/2873] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:18.744185 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:18.754341 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9fce-5bb0-43c2-892f-bd6c7fd0ac87 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:18.759443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:18.760135 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.760408 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.760671 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] PATH_INFO: `/v3/projects/08d0c2f9c0db4095b94db64602b9c0e7/users/892de79ec7ba4f779aa18fca2ac7dbd0/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.776386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d0c2f9c0db4095b94db64602b9c0e7, user_id=892de79ec7ba4f779aa18fca2ac7dbd0, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:18.783807 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:18.862148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1448/2874] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:18 2022] PUT /identity/v3/projects/08d0c2f9c0db4095b94db64602b9c0e7/users/892de79ec7ba4f779aa18fca2ac7dbd0/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:18.931643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:18.931929 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:18.932179 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:18.959870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] MFA Rules not processed for user `892de79ec7ba4f779aa18fca2ac7dbd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:18.961083 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:18.974301 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:19.027648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1427/2875] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:19.039434 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:19.042001 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:19.099072 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951aa846-3b8d-43b2-97ee-d7c8784c4379 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:19.103236 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:19.103859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:19.104201 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:19.104465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:19.112762 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:19.114209 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad80099-08d0-4880-974d-36db3e51b1ec service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:19.122951 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1449/2876] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:19.145149 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:19.211933 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d6c9e-45a9-4bcf-a171-ec96f00d9422 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:19.217321 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:19.217988 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:19.218260 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:19.218533 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] PATH_INFO: `/v3/users/892de79ec7ba4f779aa18fca2ac7dbd0` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:19.219659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=892de79ec7ba4f779aa18fca2ac7dbd0)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:19.221572 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:19.413921 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892de79ec7ba4f779aa18fca2ac7dbd0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:19.414653 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892de79ec7ba4f779aa18fca2ac7dbd0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:19.512899 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892de79ec7ba4f779aa18fca2ac7dbd0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:19.521307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892de79ec7ba4f779aa18fca2ac7dbd0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:19.534680 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892de79ec7ba4f779aa18fca2ac7dbd0'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:19.541485 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1428/2877] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:19 2022] DELETE /identity/v3/users/892de79ec7ba4f779aa18fca2ac7dbd0 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:19.546171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:19.546456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:19.546727 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:19.566445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:19.567742 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:19.581370 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:19.632355 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1450/2878] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:19.642671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:19.644271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:19.685360 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:19.702890 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af170-fa05-45f6-8755-3b0ba680d13e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:19.706512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:19.707107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:19.707381 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:19.707648 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:19.716580 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:19.718277 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff06cb-013b-4d4c-97d0-d0bf595ac9e3 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:19.729735 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1429/2879] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:20.164789 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.173873 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:20.179859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04365-a8c3-450b-bf2b-163618e14d5a None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.185443 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.186180 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.186439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.186698 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] PATH_INFO: `/v3/projects/08d0c2f9c0db4095b94db64602b9c0e7` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.188024 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d0c2f9c0db4095b94db64602b9c0e7)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.190027 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:20.326553 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d0c2f9c0db4095b94db64602b9c0e7', 'request_id': 'req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:20.332835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d0c2f9c0db4095b94db64602b9c0e7', 'request_id': 'req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:20.410738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d0c2f9c0db4095b94db64602b9c0e7', 'request_id': 'req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:20.422835 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] The token cache is being invalidate because project 08d0c2f9c0db4095b94db64602b9c0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:20.423122 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:20.423792 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-e6566bef-6e89-4b5a-849b-c5eb7ede2dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:20.425315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1451/2880] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:20 2022] DELETE /identity/v3/projects/08d0c2f9c0db4095b94db64602b9c0e7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:20.430280 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.431293 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.431418 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.452671 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:20.453934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:20.517223 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1430/2881] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:20.526996 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.527276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.527549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.549934 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:20.561124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:20.561124 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1452/2882] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:20.564486 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.575141 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b321-5d54-4cad-a532-6e49f5be982f None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.581880 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.582504 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.582789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.583061 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.583720 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:39:20.584020 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.585625 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:20.599036 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1431/2883] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:39:20.604092 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.614826 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa5d59-5ac3-4728-aacc-c1b36c743626 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.620353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.621062 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.621430 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.621961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.622679 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.624695 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:20.761331 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1453/2884] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:20 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:20.765698 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.776123 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a450b7-d181-4634-8f6d-6c85fcf708be admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.837812 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.838551 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.838833 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.839104 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.839737 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.841814 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:20.931256 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1432/2885] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:20 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:20.936152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.947465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a2a18-9b23-4c5b-8160-0e853f2ed35c admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.952187 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.952795 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.953068 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.953334 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.954033 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.955613 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:20.967581 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1454/2886] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:39:20.971754 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:20.982192 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3564e17-f6ad-4da3-b1f8-14ddf2d77b5f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:20.987639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:20.987639 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:20.987907 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:20.988132 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:20.989145 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:20.990466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269216de-ca50-4bfb-b8a8-441e35b6a927 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:21.001994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1433/2887] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:21.006466 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:21.017815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c043-9e9f-44e6-9af9-23ccd4a25e16 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:21.022461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:21.023111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:21.023386 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:21.023660 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] PATH_INFO: `/v3/projects/aa45d1589e7d4470b8d4e57a5de5750c/users/cdab1844d4f1432e80580f54f43d4b5d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:21.041326 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa45d1589e7d4470b8d4e57a5de5750c, user_id=cdab1844d4f1432e80580f54f43d4b5d, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:21.044162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:21.131046 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1455/2888] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:21 2022] PUT /identity/v3/projects/aa45d1589e7d4470b8d4e57a5de5750c/users/cdab1844d4f1432e80580f54f43d4b5d/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:21.142763 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:21.143060 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:21.143306 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:21.175836 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] MFA Rules not processed for user `cdab1844d4f1432e80580f54f43d4b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:21.177254 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:21.211870 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:21.247084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1434/2889] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:21.259526 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:21.261060 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:21.314148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:21.329888 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6b0f3-9c5d-4bae-a21e-922b361d0ecc admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:21.333977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:21.334587 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:21.334863 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:21.335132 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:21.344272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:21.346224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51299b-c1c7-4c65-be49-6e044c5ce4b3 service cinder] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:21.356883 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1456/2890] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:21.382892 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:21.458268 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c041a6d-5b46-4653-b30b-e0e6eff033de None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:21.464769 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:21.465438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:21.465743 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:21.466021 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] PATH_INFO: `/v3/users/cdab1844d4f1432e80580f54f43d4b5d` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:21.467476 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdab1844d4f1432e80580f54f43d4b5d)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:21.469740 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:21.721803 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdab1844d4f1432e80580f54f43d4b5d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:21.722469 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdab1844d4f1432e80580f54f43d4b5d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:21.831478 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdab1844d4f1432e80580f54f43d4b5d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:21.835177 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdab1844d4f1432e80580f54f43d4b5d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:21.847785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdab1844d4f1432e80580f54f43d4b5d'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:21.854375 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1435/2891] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:21 2022] DELETE /identity/v3/users/cdab1844d4f1432e80580f54f43d4b5d => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:21.858915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:21.859200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:21.859472 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:21.880118 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:21.881330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:21.944178 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1457/2892] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:21.954852 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:21.956283 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:22.016241 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f2789-b70d-4f19-ac9d-43bffb70aa89 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:22.020081 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:22.020696 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:22.021342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:22.021342 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:22.029483 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:22.031802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:22.033329 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5e41f-7dda-40e8-a4ce-24de7c2f8e19 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:22.046893 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1436/2893] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:22.543829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:22.545347 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:22.560139 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e2358-1e5e-4c90-83e7-1d1383924863 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:22.565614 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:22.566608 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:22.566891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:22.567158 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] PATH_INFO: `/v3/projects/aa45d1589e7d4470b8d4e57a5de5750c` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:22.568338 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa45d1589e7d4470b8d4e57a5de5750c)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:22.570395 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:22.710097 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa45d1589e7d4470b8d4e57a5de5750c', 'request_id': 'req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:22.717490 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa45d1589e7d4470b8d4e57a5de5750c', 'request_id': 'req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:22.815916 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa45d1589e7d4470b8d4e57a5de5750c', 'request_id': 'req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:22.827868 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] The token cache is being invalidate because project aa45d1589e7d4470b8d4e57a5de5750c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:22.828179 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:22.828821 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-9a093ab7-4ba4-4f0b-90b9-9be954ac09c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:22.830353 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1458/2894] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:22 2022] DELETE /identity/v3/projects/aa45d1589e7d4470b8d4e57a5de5750c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:22.835184 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:22.835676 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:22.835778 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:22.856352 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:22.857576 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:22.919106 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1437/2895] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:22.926808 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:22.927096 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:22.927360 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:22.947638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:22.948830 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:22.956647 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1459/2896] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:22.960868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:22.979341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c12507-8476-4ace-bc84-e80d9f99f819 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:22.990398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:22.990398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:22.990398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:22.990398 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] PATH_INFO: `/v3/domains` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:22.990810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 21:39:22.990810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:22.990810 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.000710 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1438/2897] 10.222.0.99 () {58 vars in 1153 bytes} [Fri Sep 2 21:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 21:39:23.005299 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.015765 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23162783-b155-4af8-ab55-1848511d2e27 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.020498 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.036289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.036289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.036289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.036289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.036289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.138141 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1460/2898] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:23 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:23.142730 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.153511 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97300cde-91a4-4c78-ac7b-82be0a88526f admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.210857 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.211463 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.211746 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.212124 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.213275 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.215245 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.313348 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1439/2899] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:23 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:23.317702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.329159 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1d514-99ea-4283-88cb-90102e68f045 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.333879 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.334504 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.334784 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.335057 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.335693 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.338017 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.348253 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1461/2900] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:23.352216 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.362447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33b1e6-3af0-4635-af82-db427e60cea5 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.367085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.367682 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.367958 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.368879 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.369760 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.371354 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635b016-4042-4a4f-8b62-6c4b7042ade1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.381966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1440/2901] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:39:23.387049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.397011 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44942b45-25a7-4a44-bc11-76245b74c2cd admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.401939 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.402584 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] REQUEST_METHOD: `PUT` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.402865 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.403146 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] PATH_INFO: `/v3/projects/a034f890ad2b494480db23018198fdcd/users/ccd84e78cb4e4d60ab101a608fbe814f/roles/e1924b0c1a3541d0a78b3f6ce099d6dc` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.417911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a034f890ad2b494480db23018198fdcd, user_id=ccd84e78cb4e4d60ab101a608fbe814f, role_id=e1924b0c1a3541d0a78b3f6ce099d6dc)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.420645 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.507700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1462/2902] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:23 2022] PUT /identity/v3/projects/a034f890ad2b494480db23018198fdcd/users/ccd84e78cb4e4d60ab101a608fbe814f/roles/e1924b0c1a3541d0a78b3f6ce099d6dc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:23.519046 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.519333 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.519610 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.561906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] MFA Rules not processed for user `ccd84e78cb4e4d60ab101a608fbe814f`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:23.563415 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:23.611729 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:23.628903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1441/2903] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:23.633470 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.639275 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:23.699280 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aab7db-e736-46d3-bfea-59c2605f80de admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.702513 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.703137 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.703412 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.703678 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.704312 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.706247 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:23.823800 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1463/2904] 10.222.0.99 () {60 vars in 1153 bytes} [Fri Sep 2 21:39:23 2022] POST /identity/v3/projects => generated 375 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:23.827569 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:23.839252 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f8e4d-9fb7-495f-8611-9a42b59339b3 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:23.895869 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:23.896473 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:23.896749 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:23.897015 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] PATH_INFO: `/v3/users` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:23.897646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:23.899592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.033910 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1442/2905] 10.222.0.99 () {60 vars in 1144 bytes} [Fri Sep 2 21:39:23 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 21:39:24.038252 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:24.050611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca07b65-b215-402f-9221-753d9d473f80 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:24.055200 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:24.056989 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.057274 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.057549 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.058231 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:24.059829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.070600 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1464/2906] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 21:39:24.074889 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:24.085246 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eb5e1-56df-4686-87c8-ef18e4ffc825 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:24.089934 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:24.090582 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.090861 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.091136 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] PATH_INFO: `/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.107079 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=470a78efb38b4fc6aad4f1ab0a9eb4fc, user_id=3f2ed1d0ab6f4c47ad7e8faa8495cf37, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:24.109789 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.220694 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1443/2907] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:24 2022] PUT /identity/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:24.225002 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:24.236461 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e926614-d1c6-4d6a-a52e-d246ece5cc12 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:24.296435 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:24.296953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.297226 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.297494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.298163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:24.299738 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dfe67-87cb-4b5c-9291-513edf4b6ea0 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.309891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1465/2908] 10.222.0.99 () {58 vars in 1122 bytes} [Fri Sep 2 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 21:39:24.314428 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:24.325421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db1702-011a-4da7-9b33-c7578c226c7e admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:24.330573 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:24.357464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] REQUEST_METHOD: `PUT` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.357464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.357464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] PATH_INFO: `/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37/roles/23a6ea4d1c1648259ca786cd5becc2fe` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.357464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] RBAC: Authorizing `identity:create_grant(project_id=470a78efb38b4fc6aad4f1ab0a9eb4fc, user_id=3f2ed1d0ab6f4c47ad7e8faa8495cf37, role_id=23a6ea4d1c1648259ca786cd5becc2fe)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:24.357464 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.358053 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1444/2909] 10.222.0.99 () {60 vars in 1483 bytes} [Fri Sep 2 21:39:24 2022] PUT /identity/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37/roles/23a6ea4d1c1648259ca786cd5becc2fe => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:24.367975 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] REQUEST_METHOD: `POST` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.368398 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.368822 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.398074 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.auth.core [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] MFA Rules not processed for user `3f2ed1d0ab6f4c47ad7e8faa8495cf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81842) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:24.399802 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:24.464771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:24.469915 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1466/2910] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:24.482356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:24.484041 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:24.544037 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642637ac-afae-450a-a063-f79c8285671d admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:24.547797 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba2b8b21f0454436a1958cb3abb27e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:24.548337 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:24.548615 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:24.548885 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:24.606266 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:24.607895 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850090c-7823-4b28-9fd9-d310ccc40678 service cinder] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:24.611666 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:24.617980 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1445/2911] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:27.804205 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:27.867298 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714a098-6d57-4b87-b69f-ff0fb3e88578 None None] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:27.872148 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:27.872782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:27.873059 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:27.873332 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] PATH_INFO: `/v3/users/ccd84e78cb4e4d60ab101a608fbe814f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:27.874494 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd84e78cb4e4d60ab101a608fbe814f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:27.876406 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:28.045782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd84e78cb4e4d60ab101a608fbe814f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.046394 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd84e78cb4e4d60ab101a608fbe814f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.053541 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd84e78cb4e4d60ab101a608fbe814f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.145003 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd84e78cb4e4d60ab101a608fbe814f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.145633 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd84e78cb4e4d60ab101a608fbe814f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.164422 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1467/2912] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:27 2022] DELETE /identity/v3/users/ccd84e78cb4e4d60ab101a608fbe814f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:28.169813 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:28.170350 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:28.170477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:28.189438 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] MFA Rules not processed for user `65f8e87a284d46c3becdb8085954efaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:39:28.190670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:28.249561 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1446/2913] 10.222.0.99 () {58 vars in 958 bytes} [Fri Sep 2 21:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:39:28.262249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:28.262439 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:28.329522 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a32c-9287-4280-8871-3fd85ccf230e admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:28.333155 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:28.333783 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:28.334063 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:28.338084 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:28.344035 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:28.345492 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:28.355380 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1468/2914] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:28.800721 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:28.802249 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:28.825616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e87ac-1338-4cf6-a6c4-2a99507796dd None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:28.825616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:28.825616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:28.825616 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:28.827421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] PATH_INFO: `/v3/projects/a034f890ad2b494480db23018198fdcd` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:28.827421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a034f890ad2b494480db23018198fdcd)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:28.827421 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:28.952641 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a034f890ad2b494480db23018198fdcd', 'request_id': 'req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:28.958856 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a034f890ad2b494480db23018198fdcd', 'request_id': 'req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.032715 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a034f890ad2b494480db23018198fdcd', 'request_id': 'req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.045206 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] The token cache is being invalidate because project a034f890ad2b494480db23018198fdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:29.045512 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.046140 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.047657 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1447/2915] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:28 2022] DELETE /identity/v3/projects/a034f890ad2b494480db23018198fdcd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:29.051763 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:29.061465 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0b059-3b1d-4d4e-aaea-8c41f164fd38 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:29.078368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:29.119047 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:29.119733 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:29.120194 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:29.120474 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] PATH_INFO: `/v3/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:29.121620 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2ed1d0ab6f4c47ad7e8faa8495cf37)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:29.123574 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:29.321692 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2ed1d0ab6f4c47ad7e8faa8495cf37'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.322424 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2ed1d0ab6f4c47ad7e8faa8495cf37'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.330189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2ed1d0ab6f4c47ad7e8faa8495cf37'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.402015 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2ed1d0ab6f4c47ad7e8faa8495cf37'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.402658 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2ed1d0ab6f4c47ad7e8faa8495cf37'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:29.421102 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1469/2916] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:39:29 2022] DELETE /identity/v3/users/3f2ed1d0ab6f4c47ad7e8faa8495cf37 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:39:29.841331 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:29.842875 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:29.856607 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4cfdae-9a63-4403-9fc9-c6665e5fe3c1 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:29.915423 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:29.916058 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:29.916390 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:29.916840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] PATH_INFO: `/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:29.918008 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470a78efb38b4fc6aad4f1ab0a9eb4fc)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:29.919937 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:29.944766 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:30.071823 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470a78efb38b4fc6aad4f1ab0a9eb4fc', 'request_id': 'req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:30.078549 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470a78efb38b4fc6aad4f1ab0a9eb4fc', 'request_id': 'req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:30.185499 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470a78efb38b4fc6aad4f1ab0a9eb4fc', 'request_id': 'req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:30.196712 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] The token cache is being invalidate because project 470a78efb38b4fc6aad4f1ab0a9eb4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:39:30.196994 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:30.197646 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:39:30.199741 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1448/2917] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:39:29 2022] DELETE /identity/v3/projects/470a78efb38b4fc6aad4f1ab0a9eb4fc => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:39:39.417721 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:40.290290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:39:55.599561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:55.601125 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:55.665906 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4a91-d0d4-4d51-92ed-0d5fcebb3f21 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:55.722288 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:55.723058 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:55.723582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:55.723582 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:55.725254 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:55.810271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:55.810271 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:55.824249 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1470/2918] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:39:57.036849 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:39:57.038623 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:57.051213 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4f2a9-512c-4c5a-adb0-15d2618e0a50 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:39:57.054850 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:39:57.055447 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:39:57.055726 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:39:57.056000 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:39:57.057732 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:39:57.072372 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:39:57.073840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:39:57.086001 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1449/2919] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:39:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:40:05.818117 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=74 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:40:07.079864 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=6 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:40:37.155276 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:40:37.157070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:37.168771 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191daf4c-77d3-4fb1-8c03-a782ba07711e service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:40:37.172427 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:40:37.173018 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:40:37.173295 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:40:37.173567 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:40:37.175272 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:37.184973 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:40:37.186432 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:40:37.194476 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1471/2920] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:40:47.190838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=6 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:40:55.064251 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:40:55.067385 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:55.115620 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6d7f7-1606-4752-8c92-6a1b864463bc service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:40:55.122941 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:40:55.123626 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:40:55.124242 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:40:55.124536 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:40:55.127506 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:55.150025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:40:55.150025 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:40:55.161367 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1450/2921] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:40:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:40:58.227248 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:40:58.229399 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:58.264487 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420df475-b0e7-4c22-97da-27f135e494e9 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:40:58.270705 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d95be82e9e04470902aef0d9bab1919', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:40:58.271388 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:40:58.271702 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:40:58.271977 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:40:58.274686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:40:58.288101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:40:58.290162 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:40:58.301811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1472/2922] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:40:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:41:05.155538 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=13 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:05.603664 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:05.618550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f49bf2-fbf1-426f-8a83-7825af36178f service nova] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:05.618550 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:05.618891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:05.618891 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:05.619172 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:05.620974 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:05.685779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:05.685779 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:05.694842 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1451/2923] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:41:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:41:08.296760 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=13 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:12.335611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:12.358547 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb174a46-1e5b-4b2b-9893-757089f9f0e9 service nova] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:12.362215 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:12.362838 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:12.363116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:12.363379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:12.365100 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:12.497049 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:12.498611 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:12.521344 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1473/2924] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:41:12.563005 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:12.590782 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81d2db-88ff-4a65-ba9f-e795c86f9913 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:12.595643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1564eaa8d8543faa415c09ddbdf4c01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:12.596632 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:12.597107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:12.597591 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:12.599786 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:12.628146 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:12.629629 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:12.644183 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1452/2925] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:41:17.869429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:17.871965 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:18.076617 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208839f-c4d4-4731-ae13-ce2dab264ca6 service neutron] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:18.083887 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:18.084534 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:18.084811 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:18.085082 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:18.087891 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:18.108774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:18.111080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:18.152877 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1474/2926] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:41:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:41:18.883674 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:18.896382 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8b43-7fa6-4384-ba71-c758d672d8a5 service neutron] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:18.902748 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:18.903362 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] REQUEST_METHOD: `GET` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:18.903644 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:18.903911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:18.905911 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:18.919708 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:18.921377 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:18.936115 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1453/2927] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:41:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 21:41:24.145848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:24.148626 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:24.168955 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e6f07-a325-4e09-a48f-5db60c6fd622 service placement] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:24.176224 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:24.177270 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:24.177686 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:24.177848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] PATH_INFO: `/v3/users/e9450f3f7527406ab70fcf0b64cab787` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:24.206730 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=100 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:24.215192 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9450f3f7527406ab70fcf0b64cab787)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:24.217627 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:24.467164 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9450f3f7527406ab70fcf0b64cab787'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:24.469227 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9450f3f7527406ab70fcf0b64cab787'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:24.476251 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9450f3f7527406ab70fcf0b64cab787'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:24.587404 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9450f3f7527406ab70fcf0b64cab787'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:24.588168 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9450f3f7527406ab70fcf0b64cab787'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:24.608961 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1475/2928] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:24 2022] DELETE /identity/v3/users/e9450f3f7527406ab70fcf0b64cab787 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:24.939099 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:24.952011 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8084d5-4297-49c8-9f05-6deca9ac75d0 service placement] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:25.020271 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:25.020949 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:25.021205 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:25.021440 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] PATH_INFO: `/v3/projects/beb5b0e674994eabb1e18fde391c2484` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:25.033493 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:25.035055 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:25.039670 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb5b0e674994eabb1e18fde391c2484)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:25.041884 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:25.055859 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed859d2-d317-4730-ba65-19ab8f2b3007 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:25.061485 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:25.062241 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:25.062649 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:25.062999 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] PATH_INFO: `/v3/users/7984fc2a59704c10905656ae32dc68af` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:25.064419 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7984fc2a59704c10905656ae32dc68af)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:25.066429 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:25.175199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb5b0e674994eabb1e18fde391c2484', 'request_id': 'req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.177075 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:25.183477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb5b0e674994eabb1e18fde391c2484', 'request_id': 'req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.336156 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb5b0e674994eabb1e18fde391c2484', 'request_id': 'req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.348307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] The token cache is being invalidate because project beb5b0e674994eabb1e18fde391c2484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:25.348593 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.349230 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.350598 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7984fc2a59704c10905656ae32dc68af'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.351728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1454/2929] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:24 2022] DELETE /identity/v3/projects/beb5b0e674994eabb1e18fde391c2484 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:25.352051 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7984fc2a59704c10905656ae32dc68af'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.355471 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:25.356854 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:25.360070 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7984fc2a59704c10905656ae32dc68af'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.423818 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff889b-4e5f-40fc-99e6-921f3c4bb0a8 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:25.428612 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:25.429868 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:25.430162 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:25.430430 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] PATH_INFO: `/v3/users/2f6b9c0532904f899ab6266e7b65a7f5` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:25.444199 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6b9c0532904f899ab6266e7b65a7f5)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:25.446477 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:25.447782 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7984fc2a59704c10905656ae32dc68af'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.448441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7984fc2a59704c10905656ae32dc68af'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.470044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1476/2930] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:25 2022] DELETE /identity/v3/users/7984fc2a59704c10905656ae32dc68af => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:25.702307 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6b9c0532904f899ab6266e7b65a7f5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.702906 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6b9c0532904f899ab6266e7b65a7f5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.725988 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:25.731351 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:25.781214 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:25.799628 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14419b0f-c295-4a31-8dd9-94591b3a632a admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:25.804829 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:25.805602 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:25.805922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:25.806219 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] PATH_INFO: `/v3/projects/ed9e8bc2976f4e2a8f439f89440bdaa3` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:25.807445 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9e8bc2976f4e2a8f439f89440bdaa3)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:25.809431 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:25.809965 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6b9c0532904f899ab6266e7b65a7f5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.817811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6b9c0532904f899ab6266e7b65a7f5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.831085 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6b9c0532904f899ab6266e7b65a7f5'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.837903 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1455/2931] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:25 2022] DELETE /identity/v3/users/2f6b9c0532904f899ab6266e7b65a7f5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:41:25.958533 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9e8bc2976f4e2a8f439f89440bdaa3', 'request_id': 'req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:25.967564 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9e8bc2976f4e2a8f439f89440bdaa3', 'request_id': 'req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.101438 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9e8bc2976f4e2a8f439f89440bdaa3', 'request_id': 'req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.112376 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] The token cache is being invalidate because project ed9e8bc2976f4e2a8f439f89440bdaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:26.112688 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.113330 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.114886 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1477/2932] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:25 2022] DELETE /identity/v3/projects/ed9e8bc2976f4e2a8f439f89440bdaa3 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:41:26.119592 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:26.121050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:26.164577 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:26.166417 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:26.196982 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:26.216042 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab5266-e12c-4ab4-a306-d9da848c20f9 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:26.251700 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2b8d0-bf8b-43eb-81b7-51c1959cf07f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:26.257101 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:26.258335 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:26.258638 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:26.258907 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] PATH_INFO: `/v3/projects/234c29fb02204981b2f2fb131b95282f` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:26.271093 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234c29fb02204981b2f2fb131b95282f)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:26.274315 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:26.286659 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:26.287559 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:26.287831 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:26.288100 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] PATH_INFO: `/v3/users/3d3e5daa8cb449c38e9e34d8434367ce` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:26.289466 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3e5daa8cb449c38e9e34d8434367ce)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:26.291522 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:26.589556 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3e5daa8cb449c38e9e34d8434367ce'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.590202 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3e5daa8cb449c38e9e34d8434367ce'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.591208 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234c29fb02204981b2f2fb131b95282f', 'request_id': 'req-8b571e2a-fa81-4d27-abe0-63c7b71e2750'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.598369 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234c29fb02204981b2f2fb131b95282f', 'request_id': 'req-8b571e2a-fa81-4d27-abe0-63c7b71e2750'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.707785 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3e5daa8cb449c38e9e34d8434367ce'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.717334 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3e5daa8cb449c38e9e34d8434367ce'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.730376 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-50401a39-29c4-4299-a2a6-065e9076738c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3e5daa8cb449c38e9e34d8434367ce'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.736545 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1456/2933] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:26 2022] DELETE /identity/v3/users/3d3e5daa8cb449c38e9e34d8434367ce => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:26.737806 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] REQUEST_METHOD: `POST` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:26.738089 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:26.738361 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:26.761180 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.auth.core [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] MFA Rules not processed for user `b8c976a6a3bf48a6a518bdd7b604087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 21:41:26.762443 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:26.806016 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234c29fb02204981b2f2fb131b95282f', 'request_id': 'req-8b571e2a-fa81-4d27-abe0-63c7b71e2750'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.815144 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:26.821170 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] The token cache is being invalidate because project 234c29fb02204981b2f2fb131b95282f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81842) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:26.821456 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.822130 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:26.829757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1478/2934] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:26 2022] DELETE /identity/v3/projects/234c29fb02204981b2f2fb131b95282f => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:41:26.830290 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1457/2935] 10.222.0.99 () {58 vars in 967 bytes} [Fri Sep 2 21:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 21:41:26.876681 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:26.878262 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:26.963512 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b571e2a-fa81-4d27-abe0-63c7b71e2750 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:27.041697 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e95d57ba23a424e8ce7e7e1695cd8a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:27.043896 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:27.044564 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:27.046231 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:27.049396 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:27.049890 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:27.052340 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:27.080019 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:27.083000 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:27.084757 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:27.101292 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1479/2936] 10.222.0.99 () {60 vars in 1384 bytes} [Fri Sep 2 21:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 21:41:27.138744 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa4df1-16db-472c-9496-3e99271ff8c7 None None] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:27.144302 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:27.144824 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:27.145107 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:27.145444 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] PATH_INFO: `/v3/projects/17302b86015842cfab74aacb763b0130` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:27.147728 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17302b86015842cfab74aacb763b0130)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:27.149225 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:27.288067 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17302b86015842cfab74aacb763b0130', 'request_id': 'req-f16b044c-4199-431b-9251-15bbce4abe26'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:27.294783 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17302b86015842cfab74aacb763b0130', 'request_id': 'req-f16b044c-4199-431b-9251-15bbce4abe26'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:27.467802 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17302b86015842cfab74aacb763b0130', 'request_id': 'req-f16b044c-4199-431b-9251-15bbce4abe26'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:27.479624 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] The token cache is being invalidate because project 17302b86015842cfab74aacb763b0130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:27.480166 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:27.480945 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:27.483113 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1458/2937] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:27 2022] DELETE /identity/v3/projects/17302b86015842cfab74aacb763b0130 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 21:41:36.495171 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:36.496734 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:36.511823 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca74d36-689f-4b80-ad1b-273334626f87 service swift] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:36.572478 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:36.573023 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:36.573325 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:36.573561 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] PATH_INFO: `/v3/users/43fefdc28eb54c9f8401bf11b8935065` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:36.574806 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fefdc28eb54c9f8401bf11b8935065)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:36.576899 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:36.780953 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:36.794337 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fefdc28eb54c9f8401bf11b8935065'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:36.795014 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fefdc28eb54c9f8401bf11b8935065'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:36.805911 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fefdc28eb54c9f8401bf11b8935065'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:36.875368 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fefdc28eb54c9f8401bf11b8935065'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:36.876154 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fefdc28eb54c9f8401bf11b8935065'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:36.895774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1480/2938] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:36 2022] DELETE /identity/v3/users/43fefdc28eb54c9f8401bf11b8935065 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:37.198297 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:37.200635 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:37.216095 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b044c-4199-431b-9251-15bbce4abe26 admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:37.279007 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:37.279642 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:37.279920 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:37.280189 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] PATH_INFO: `/v3/projects/634952274d2748b988c4610754edc371` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:37.281305 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=634952274d2748b988c4610754edc371)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:37.283264 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:37.448247 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634952274d2748b988c4610754edc371', 'request_id': 'req-a87731e0-2dd9-403d-ac0e-073507a3df0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.450084 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:37.454373 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634952274d2748b988c4610754edc371', 'request_id': 'req-a87731e0-2dd9-403d-ac0e-073507a3df0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.560966 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634952274d2748b988c4610754edc371', 'request_id': 'req-a87731e0-2dd9-403d-ac0e-073507a3df0c'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.572356 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] The token cache is being invalidate because project 634952274d2748b988c4610754edc371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:37.572643 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.573313 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.574811 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1459/2939] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:37 2022] DELETE /identity/v3/projects/634952274d2748b988c4610754edc371 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:37.578922 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:37.580462 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:37.593313 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6157e6-3912-4c57-9a53-12f9f0259c5f admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:37.657848 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:37.658496 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:37.658774 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:37.659044 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] PATH_INFO: `/v3/users/f52b32d40cde4c48ab20cd9e50d21fd5` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:37.660189 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52b32d40cde4c48ab20cd9e50d21fd5)` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:37.662152 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:37.902291 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52b32d40cde4c48ab20cd9e50d21fd5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.903054 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52b32d40cde4c48ab20cd9e50d21fd5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:37.911421 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52b32d40cde4c48ab20cd9e50d21fd5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.001149 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52b32d40cde4c48ab20cd9e50d21fd5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.001815 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.notifications [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52b32d40cde4c48ab20cd9e50d21fd5'} {{(pid=81842) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.021895 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1481/2940] 10.222.0.99 () {58 vars in 1224 bytes} [Fri Sep 2 21:41:37 2022] DELETE /identity/v3/users/f52b32d40cde4c48ab20cd9e50d21fd5 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:38.293408 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Authenticating user token {{(pid=81843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:41:38.295050 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.fernet_utils [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:41:38.310784 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87731e0-2dd9-403d-ac0e-073507a3df0c admin admin] Validating token access rules against request {{(pid=81843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:41:38.374434 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65f8e87a284d46c3becdb8085954efaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8077ff7239e7425ab3b0ae8d4fecdd2f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:41:38.375068 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:41:38.375341 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:41:38.375609 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] PATH_INFO: `/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df` {{(pid=81843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:41:38.376807 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea2f03184ce48ffa7b6b3f8c7cf75df)` {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:41:38.378742 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] RBAC: Authorization granted {{(pid=81843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:41:38.576130 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea2f03184ce48ffa7b6b3f8c7cf75df', 'request_id': 'req-e004020d-9286-43d1-abe4-67cc80bd0de2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.582493 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea2f03184ce48ffa7b6b3f8c7cf75df', 'request_id': 'req-e004020d-9286-43d1-abe4-67cc80bd0de2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.749774 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea2f03184ce48ffa7b6b3f8c7cf75df', 'request_id': 'req-e004020d-9286-43d1-abe4-67cc80bd0de2'} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.761791 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] The token cache is being invalidate because project cea2f03184ce48ffa7b6b3f8c7cf75df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 21:41:38.764208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.764208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG keystone.notifications [None req-e004020d-9286-43d1-abe4-67cc80bd0de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 21:41:38.764208 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: [pid: 81843|app: 0|req: 1460/2941] 10.222.0.99 () {58 vars in 1233 bytes} [Fri Sep 2 21:41:38 2022] DELETE /identity/v3/projects/cea2f03184ce48ffa7b6b3f8c7cf75df => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 21:41:48.018080 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:41:48.757840 dv-cinder-852211-9-iscsi devstack@keystone.service[81843]: DEBUG dbcounter [-] [81843] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 21:43:41.473379 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Authenticating user token {{(pid=81842) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 21:43:41.474904 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:43:41.569116 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb841ea5-166d-48c8-9ccb-e2b399fcf0e1 admin admin] Validating token access rules against request {{(pid=81842) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 21:43:41.636936 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af4ca265da3485cb7cec625913274ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3c5c8aceb40a8be5c0864c4ec89ba', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81842) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 21:43:41.637571 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] REQUEST_METHOD: `GET` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 21:43:41.637892 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] SCRIPT_NAME: `/identity` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 21:43:41.638163 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81842) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 21:43:41.640066 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.fernet_utils [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81842) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 21:43:41.653441 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 21:43:41.655111 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9088f4-b7ab-4525-9bc9-efa3eb47c633 service placement] RBAC: Authorization granted {{(pid=81842) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 21:43:41.666289 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: [pid: 81842|app: 0|req: 1482/2942] 10.222.0.99 () {60 vars in 1365 bytes} [Fri Sep 2 21:43:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 21:43:51.661449 dv-cinder-852211-9-iscsi devstack@keystone.service[81842]: DEBUG dbcounter [-] [81842] Writing DB stats keystone:SELECT=67 {{(pid=81842) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}